Multimedia document annotation is used in traditional multimedia database systems. However, without the help of human beings, it is very difficult to extract the semantic content of multimedia automatically. On the other hand, it is a tedious job to annotate multimedia documents in large databases one by one manually. This paper first introduces a method to construct a semantic network on top of a multimedia database. Second, a useful and efficient annotation strategy is presented based on the framework to obtain an accurate and rapid annotation of any multimedia databases. Third, two methods of joint similarity measures for semantic and low-level features are evaluated.