Endoscopy provides a direct evaluation of mucosal lesions in inflammatory bowel disease (IBD), permitting the description of elementary lesions, their surface extent and severity. The severity of mucosal lesions directly reflects disease activity and may help to identify an aggressive behavior of the disease. Several studies have recently pointed out the potential role of endoscopy in the prediction of IBD outcome. Indeed, severe endoscopic lesions in Crohn's disease (CD) patients, defined by deep and extensive ulcerations on at least one part of the colon, are associated with an increased risk of penetrating complication and surgery. Severe endoscopic lesions during severe attacks of ulcerative colitis (UC) are associated with an increased risk of colectomy in the short and long term. Severity of postoperative recurrence in CD may help to predict the risk of clinical relapse and need for further surgery. Achievement of mucosal healing, which can be obtained by administration of several types of drugs, is associated with a better outcome, less surgery and hospitalization. This review focuses on the assessment of endoscopic severity in CD and UC and on the impact of endoscopic severity on disease outcome. More specifically, we discuss how endoscopy can be used at different stages of IBD to predict the disease course and/or to adapt treatment strategies.