The perceived quality of fruits, such as mangoes, is greatly dependent on many parameters such as shape, size, ripeness, firmness, aroma flavour, and is influenced by other factors such as harvesting time. Unfortunately, a manual fruit grading has several drawbacks such as subjectivity, tediousness, cost, inefficiency, and inconsistency. By automating the procedure, as well as developing new classification technique, may solve these problems. Currently, there have been a number of reported results on the use of non-destructive techniques to evaluate and determine mango maturity and ripeness levels. However, most of these reported studies were conducted using a single-modality sensing systems. This paper presents the work on the multi modality sensing system for classification of quality for mangoes (Magnifera Indica cv. Harumanis) using electronic nose (e-nose), acoustic sensor, CCD camera (visible) and IR camera (non visible). A Fourier-based shape separation method was developed from CCD (visible) data to grade mango by its shape and the results was able to correctly classify 98% by shape. Colour intensity from infrared image was used to distinguish and classify the level of ripeness of the fruits based on the colour spectrum and the results classify 92% by maturity levels. It also can be used to predict infested fruits by fungus, weevil, blemish, fruit fly and many more. E-nose was trained to detect the differences in aromas and acoustic sensor detect on juiciness of the mango samples. The results shows that the entire sensors can determine difference quality parameters of fruits and it was also observed that fusion of CCD camera, IR camera, E-nose and acoustic sensor data provides a real alternative to human expert panel in non-destructive fruit quality assessment for mango. The performances were compared with human panels and the results show that this technique achieved similar accuracy.