This research work presents an image-based real-time facial expression recognition system that can recognize six（6）distinct facial expressions efficiently from a normal camera and videos without loosing the generalization ability.In particular,we implement a real-time facial expression recognition system that leverages a new CNN model and achieves a better performance than previous implementations.Virtual sampling was adopted in this research work to improve the representation of classes with fewer data through re-sampling and adding some noise to the other samples.In addition,we also consider the concept of building a new facial expression dataset to solve the problem of over-fitting due to lack of insufficient training data size of facial expression datasets.To reduce the over-fitting of our models further and increase accuracy as well,We utilized various approaches including dropout method and batch normalization in addition to Adam optimization technique.The proposed CNN model has been validated on the three publicly available datasets（CK+,KDEF and Oulu-CASIA VIS datasets）and our private dataset comprising six（6）emotions（surprise,anger,disgust,fear,happiness,and sadness）.These four datasets were merged,shuffled and trained together and an average test accuracy of 97.24%was achieved.The validation on the test set using confusion matrix reveals that theproposed CNN model achieves better performance in comparison to the other traditional methods.