In recent years, Convolutional Neural Networks (CNN) have been applied to a wide range of computer vision tasks thanks to their ability to learn informative features from data without human supervision. CNN models have been successfully utilized to segment and classify natural images where there are large datasets, nonetheless, many structures have been proposed to improve their performances. By contrast, in the medical field even though datasets are smaller than their natural images counterpart, CNN models have also over-perform previous methods and are becoming the first choice to tackle a variety of probleMaster. However, it is unclear how CNN structures are able to make a better use of the limited training data to generalize and produce reliable predictions. Therefore, in this study a set of CNN structures was analyzed on healthy brain tissue segmentation as well as brain abnormalities segmentation to understand the effect of the architecture design and the loss function used to train the model. The main contributions of the present thesis are the proposed CNN architecture that is capable of segmenting the brain across different tasks, the proposed loss function that increases the accuracy and reduces the number of false negatives and the segmentation quality analysis across different metrics that allow us to understand the advantages and disadvantages of a CNN model.