Even during the pandemic, mangosteen is still the prima donna of foreign market demand and the highest exotic fruit export commodity in Indonesia. Only good quality mangosteen that meets the requirements can be exported, including the factor of mangosteen peel defect. Therefore, in this research, rapid grading was carried out to measure the level of mangosteen peel defect by distinguishing the skin texture into 3 levels, namely Fine (F), Medium (M), and Rough (R). Image data retrieval using 300 mangosteens with different maturity levels, where 240 image data for training and 60 image data for testing. Until 20 features of texture extraction based Gray Level Co-occurrence Matrix (GLCM) has been evaluated. Because of non-iterative weighted learning (non-epoch), the Extreme Learning Machine (ELM) can be forced to find the optimum performance by the number of hidden neurons. The results show that the training performance value reaches 99.58%, the testing performance value reaches 96.67%, and the 5-fold cross-validation accuracy until 97%. With the training time is under 0.3 seconds, so the extreme learning machine is very suitable for rapid grading applications.