Within the Machine Learning/ Neural Network community there is a large consent that in order to have a well trained Neural Net one must use a large, diverse set of data. This data should be annotated in some way such that the way to identify them is known. Using this* paper I recreated there Neural network using Keras. Their architecture consists of “a contracting path to capture context and a symmetric expanding path that enables precise localization”. The results show that this network can be well trained to segment an image using a small amount of data.
My code for this implementation can be found here