Semantic segmentation on an image is increasingly required in more and more fields such as scene understanding, inference of object relationships for autonomous driving and object extraction of interest. This technique gives the ability to segment different parts and objects from an image. Recently, there have also been many improvements in segmentation based on deep learning techniques. In particular, SegNet has improved the noisy image from the results of pixel-wise labelling. In this paper, a deep learning method for the segmentation of each area in an image is proposed. We introduce a progressive growing of convolutional neural networks that can learn quickly and increase the recognition rate using various resolutions. Then we compare our results to the learning methods using conventional convolutional neural networks architecture. We also show that our segmentation network works in real-time.