A 3D surface modeling format that is commonly used today is point cloud. 3D surface model segmentation could provide data for analysis in various fields. In the context of Geographic Information Systems, point cloud data obtained from the Light Detection and Ranging (LiDAR) sensor are used by machines to automatically identify objects such as houses, buildings, land, and rivers. There has been many Deep Learning approach through Convolutional Neural Network (CNN) that has been proven to be very capable for 2-dimensional imagery classification and segmentation. PointNet is a Deep Learning architecture that is designed so that the point cloud format that is still tabular form, can be directly convoluted by the CNN model. In this study, an improvement of PointNet is proposed for Point Cloud data of Kupang City. The Point Cloud data were acquired using an Unmanned Aerial Vehicle with a LiDAR sensor installed. The data were pre-processed and divided into training and testing data. The data were processed with the PointNet architecture and the model was tested using several metrics. The experiment shows that the PointNet architecture is capable on segmenting Geographical Point Cloud Data. In addition, incorporating voxel's color features could increase the performance of the segmentation.