论文部分内容阅读
Abstract: Through the analyses of both quantitative and qualitative data, the present study mainly aims to explore the possibility of Listening-for-Summary (LS) as an alternative listening comprehension test format in Practical English Test for Colleges (PRETCO).
Key Words: listening-for-summary, listening comprehension, test format, PRETCO
1．Introduction
The objectives for higher vocational and technical education are to train talented technicians and staffers in the front line of production, service, management, etc. The students graduating from those colleges will go to work immediately. They may use English to take a message over the telephone, take part in talks, meetings or lectures and then write a report, etc. These activities have integrated two or more of the four skills, that is, listening, speaking, reading and writing. However, many graduates are still unable to understand lengthy talks in English on their working sites in spite of 6 or 7 years' study of English. Moreover, some of them can not apply what they have learned into practical work, especially when foreign languages are needed.
Evidently, one of the weakest ability components in their use of English in practical situation is that they cannot understand foreigners' lengthy talks so that they cannot make relevant responses.
To that end, we are proposing an item format for testing listening comprehension, namely, listening-for-summary. By this format, listening comprehension and summary writing are integrated because we believe that in many real-life work situations the technicians, mechanics, staffers, managers, etc., are very likely required to give short but clear reports in foreign languages, either spoken or written, to their superiors, or give briefings to their subordinators, after they have attended training, lectures, negotiation, etc. We also believe that such a test format will probably exert positive effect on the relevant teaching. Therefore, we have conducted a preliminary study on this format for the possibility of using it in the Practical English Test for Colleges (PRETCO).
2. Test Specifications of the PRETCO-LS
A test's specifications provide the official statement about what the test tests and how it tests it. In the following part, only the items which are different from the current listening comprehension part in PRETCO are mentioned.
2.1 Test operations
A summary table of the operations that the PRETCO-LS intends to test is below:
A Explicit meaning comprehension
01 distinguishing main points from supporting details
02 listening for the explicitly stated main ideas
B Implied meaning comprehension
03 synthesizing the input
C Contributory meaning comprehension
04 recognizing phonological features
05 recognizing structural and grammatical features
D Enacting skills
06 reducing what is heard to an outline of main points and important specifics
2.2 Performance conditions
Textual conditions
Texts of the PRETCO-LS will mainly be selected from subject-matter materials which relate to business environment but no specialty knowledge is required. All vocabulary in the test is included in the syllabus. Core vocabulary will be adopted and no technical words. Considering this kind of listening comprehension test format, it is reasonable to choose lengthy materials. Nevertheless, choosing too long materials is also unsatisfactory. One obvious reason is listener fatigue; another is the effect of memory load. So a discourse of about 200-300 words in length is believed to be appropriate.
Task conditions
The format is a listening for summary writing. A selected essay or conversation of about 200-300 words is given. After listening to the essay or conversation, testees are required to write a summary of about one third or fourth of the length of the original material. The material is repeated three times. The text lasts three to four minutes. The total time available for the test is about 15 minutes, including the time to read instructions, deliver the texts and a pause for completing task. Rubrics in English will be presented both aurally on the tape and in written on testees' paper. Testees will be reminded to listen and read the rubrics carefully before performing the task. The speech rate of the test is about 110 wpm, which is slightly slower than 120 wpm stated in the syllabus. The obvious reason is that the testees are not familiar with this new listening comprehension testing format. There is no background noise in the input material of the PRETCO-LS.
Marking system
A.The PRETCO is to test whether testees have reached the requirements stated in the syllabus of the PRETCO.
B.The PRETCO-LS adopts holistic and analytic scoring. First, the markers give reward scores according to the general impression of testees' responses; secondly, the markers give reward scores according to the key points of the summary, which have been clearly specified in the marking scheme. (See Table 1.)
C.Each summary is scored by two markers. Where discrepancies are small, the two scores can be averaged; where they are larger, a third marker will be called in and then the three scores are averaged for the final.
D. Scoring is based on the accuracy and completeness of the information explicitly stated in the testees' summaries. It is not based on mechanics or writing style.
E. It is probably worth mentioning again that in the scoring there is no reason to deduct points for errors of grammar or spelling provided testees can express what they want to express.
3. Pilot test
3.1 Sample
The test was conducted in Shanghai College of Electricity & Machinery Technology. A total of 153 second-year students attended the test. They were selected from a wide range of subject areas including technology/engineering, information/science, economy/management, machinery. All of them took the PRETCO in June 2004. Their scores of the PRETCO were obtained from the test committee so that the correlation studies between the PRETCO-LS and the PRETCO-LC could be conducted. 13 students' scores of the PRETCO were unfit and therefore were discarded. So the actual size of the testee sample for the main trial is 140.
3.2 Administration
The test was administered in five language labs in Shanghai College of Electricity & Machinery Technology. In each lab, two invigilators were assigned to prevent the act of cheating.
4. Data analysis
4.1 Quantitative data analysis
4.1.1 Data descript_ion
Data analysis was made on the PRETCO-Listening Comprehension (PRETCO-LC) and the PRETCO-Listening for Summary (PRETCO-LS) scores of the 140 testees.
The most interesting finding from the descript_ive analysis (see Table 2) is that the standard deviation of the PRETCO-LS scores is wider than that of PRETCO-LC. This result confirms us in the belief that the PRETCO-LS can discriminate testees better than PRETCO-LC does.
In addition, we can find the mean of the PRETCO-LS is lower than that of PRETCO-LC. This difference may be resulted from the fact that most of testees have been used to the Multiple Choice Question (MCQ) format that is employed in the first two sections of the listening comprehension part in PRETCO-LC and are not familiar with the new format used in the PRETCO-LS. Another reason is probably due to the fact that it is comprehensible. In PRETCO-LC, the first two sections belong to MCQs, students can guess the correct answers although they do not understand what they have heard. Moreover, the four choices within each question can give students some clues so that it can help them understand what is going on. Most of testees got most of marks or full marks in first two sections while lost marks in section 3 which required the productive ability. While in the PRETCO-LS, it focuses on "listening and writing" operations, which, to a larger degree, integrate knowledge about vocabulary, grammar, structure and skills in organizing language into listening comprehension, so it is natural for this test to reflect a comprehensive language competence to some extent.
As we refer to the two histograms of the PRETCO-LS and the PRETCO-LC score (see Figure 1), we can see both results fall in a normal curve, which shows that the sample has very good representativeness.
Figure 1 The PRETCO-LC and the PRETCO-LS histograms
4.1.2 Correlation between PRETCO-LS and PRETCO-LC
After examining the results of the 140 testees, we found 9 sets of scores are obviously abnormal. The 9 testees' scores in the PRETCO-LS are much lower than their scores in the PRETCO-LC, with the differences ranging from 7 to 10. Therefore the scores they obtained in the PRETCO-LS are not credible for the data analysis. It is reasonable to discard them. Then correlation studies were conducted based on the valid samples (131 students). The correlation coefficient between the two tests is .635 which is significant at the 0.01 level (see Table 3)
**Correlation is significant at the 0.01 level (2-tailed).
The relatively high correlation between the PRETCO-LS and the PRETCO-LC gives us some confidence to say that, to a large extent, what the PRETCO-LS tests is measuring of listening competence. However, the correlation is not so high as to render the two tests identical in their focuses of testing.
4.2 Qualitative data analysis
Along with the main trial of the PRETCO-LS, a questionnaire survey was conducted to students who took the test. In addition, another questionnaire survey was conducted among 43 teachers coming from the Foreign Language Department of Shanghai College of Electricity & Machinery Technology and all of them have rich experience in training students for sitting for the PRETCO. It should be pointed out that most questions in the questionnaire allow one answer only, but some allow more than one answer. Moreover, some respondents leave questions unanswered occasionally.
4.2.1 Questionnaire to testees
The aim of the questionnaire is to obtain the testees' feedback to the following information:
●Their general attitudes to the PRETCO-LS and the PRETCO-LC
●Their attitudes to the formats
●Whether the timing was sufficient for completing the task
●Whether the delivery rate is appropriate
●Whether the length of the text is appropriate
●Their attitudes to the usefulness of the test for their future job
●The difficulties encountered in comprehending the input
The feedback from the questionnaire is desirable on the whole. 72% of the testees believe the PRETCO-LS is a good measurement of their competence in listening comprehension. Majority of the testees agree that the test conditions such as speech rate and test time are appropriate, and the test can measure the ability to understand spoken English in their future job. Concerning the test formats used in the PRETCO-LS and the PRETCO-LC, the results show that the short dialogue multiple choice format is preferred by most students. Though 89% of the testees rated the format used in the PRETCO-LS as the least preferred, the result of the correlation study shows that this test has a medium-high correlation with the PRETCO-LC. The probable reason for the testees' negative feedback on this format is that the testees were not confident in their performance of the task because the task requires the testees not only to understand the spoken discourse but also to extract the main points and organize them into a summary. Many testees feel such a test task is very demanding. As for the problem they encountered in the PRETCO-LS, distinguishing main points from the details and organizing the summary were rated the most difficult by the testees. This finding proves that the score of the PRETCO-LS can be used to make decision on whether a testee is good or not at listening comprehension.
4.2.2 Questionnaire to teachers
The teachers' questionnaire has many questions in common with those in the students' questionnaire. Additionally, this survey provides us with some other information.
●Whether some reforms should be taken for the PRETCO-LC format
●Their attitudes to the overall design of the test paper
●Their attitudes to the difficulty of the PRETCO-LS
●Whether the marking scheme is reasonable
In general all the teachers gave a very positive response on the overall validity of the PRETCO-LS and the PRETCO-LC. 81% of the teachers think it is necessary to reform the listening comprehension test format of the PRETCO. When asked about their evaluation of the overall design of the PRETCO-LS, 85% of the teachers consider it good or fairly good. Interestingly, teachers' opinions differ greatly from those of the students in the preference of test formats. The “short-dialogue multiple choice” task is the most unpopular with them, followed by the “conversation comprehension multiple choice” task. “compound dictation”,“passage listening comprehension” “short answer question”and “listening for summary” tasks gain most support from these teachers. From the view of teachers, the subjective formats encourage the use of various types of listening comprehension operations, hence resulting in a positive washback effect on teaching. 88% of the teachers are satisfied with the marking scheme of the test.
5. Conclusions and improvements
To sum up, PRETCO-LS has achieved a high validity and reliability with a fairly high correlation with the PRETCO-LC. The correlation shows that the same construct is being tested by the two listening tests. Of course, the correlation coefficient is not higher than .70, which is actually not desirable in this situation because we believe the two formats should not be identical to each other and some new tasks and new operations are introduced. Moreover, the format gains general credit from both teachers and students. It is believed that the test is focusing on the measurement of listening competence rather than any other language traits.
Though our research has produced a preliminary framework for designing an alternative listening test format for the PRETCO, we believe that due to the restrictions in time, human resources, finance, and the researcher's limit capacity of the knowledge of language testing, there must remain much room for further improvement and that there are many issues waiting for other researchers to deal with.
References
[1]Baker, D. 1989. Language Testing. Edward Arnold Ltd.
[2]Buck, G. 2001. Assessing Listening. Cambridge: Cambridge University Press.
[3]Carroll, B.J.1980. Communicative Performance Testing. Pergamon Press Ltd.
[4]Carroll, J. B. & Hall, P. 1985. Write Your Own Language Tests. Oxford: Pergamon Institute of English.
[5]刘鸿章，1997，《重视实用能力的考核》， 《外语界》， 第四期。
[6]金艳，2002，《高级英语阅读测试的开发和效度研究》，上海：上海交通大学出版社。
[7]教育部高等教育司，2000， 《高职高专教育英语课程教学基本要求》（试行），北京：高等教育出版社。
虞苏美，杨乾龙，2001，《商务英语听力》，北京：高等教育出版社。
Appendix 1
考生问卷调查
1．你认为目前的PRETCO听力测试能在什么程度上客观，真实地反映你的英语听力水平？
(12%)A很大程度 (58%) B一般程度 (20%)C 较小程度 (0%)D不能
2．你认为目前的PRETCO听力测试语速是否合适？
(29%)A太快(64%) B 正好 (7%)C太慢
3．你认为目前的PRETCO听力测试答题时间是否合适？
(16%)A 太长 (65%)B正好 （19%）C太短
4．你认为刚才完成的听力考试在什么程度上真实地反映了你的英语听力水平？
(24%)A 很大程度(48%)B一般程度 (28%)C较小程度(0%)D不能
5．你认为刚才完成的听力考试的语速是否合适？
(24%)A 太快(71%)B正好 (5%)C太慢
6．您认为刚才完成的听力考试中文章的长度是否合适？
（3%）A 太长（14%）B 长 （73%）C正好 （8%）D 短（2%）E 太短
7．你较喜欢的听力题题型为：（可多选）___________
(92%)A 短对话多项选择题
(45%)B 长对话多项选择题
(12%)C短文理解多项选择题
(5%)D复合式听写
(3%)E 间答题
(13%)F 听力摘要（Listening for Summary）
8．你不喜欢的听力题题型为：（可多选）___________
(3%)A 短对话多项选择题
(56%)B 长对话多项选择题
(88%)C短文理解多项选择题
(95%)D复合式听写
(93%)E 间答题
(89%)F 听力摘要（Listening for Summary）
9．你认为完成刚才的听力考试，时间是否合适？
(1%)A很长 (13%)B较长(74%)C. 适中(12%)D. 较短(0%)E. 很短
10．你对这种听力题型熟悉吗？
(2%)A 很熟悉 (33%)B较熟悉 (41%)C听说过 （24%）D从来没有听说过
11．你觉得这种题型对你将来的工作会不会有用？
（17%）A很有用（72%）B可能有用 (11%)C 不会有用
12.在本次考试听力过程中，你遇到哪些困难？(可多选)
(14%)A. 语速太快 (36%)E无法把握重点
(55%)B词汇太难 (87%)F听懂但不会组织语言
(65%)C句子结构复杂 G其他___________
(42%)D题型不熟悉
Appendix 2
教师问卷调查
1．您认为目前的PRETCO 的听力测试能在什么程度上真实地反映考生的英语听力水平？
（28%）A.很大程度 （68%）B.一般程度
（4%）C.较小程度（0%）D.不能
2．您对目前的PRETCO听力考试内容题材是否喜欢？
（12%）A. 很喜欢（45%）B.较喜欢
（32%）C.一般 （1%） D.不喜欢
3．您对目前的PRETCO听力考试的题型是否喜欢？
（5%）A. 很喜欢（32%）B.较喜欢
（63%）C.一般(0%)D.不喜欢
4．您认为是否有必要改革现在的听力题型？
(24%)A. 很有必要 (57%)B.较有必要
(19%)C.不必要
5．您认为本次考试(The PRETCO-LS)能在什么程度上反映学生的英语听力水平？
(23%)A.很大程度(69%)B.一般程度
(8%)C.较小程度(0%)D.不能
6. 您对此次听力考试试卷的总体设计评价如何？
（5%）A很好(67%)B 较好
（28%）C一般（0%） D不好
7．您较喜欢的听力题型是 (可多选)___________
(6%)A. 短对话选择题（82%）D. 复合式听写
(21%)B. 长对话选择题 （79%）E. 间答题
(58%)C. 短文理解选择题 （76%）F.听力摘要
8．您不喜欢的听力题型是（可多选）___________
（83%）A. 短对话选择题（2%）D. 复合式听写
（45%）B. 长对话选择题（5%）E.间答题
（11%）C. 短文理解选择题（7%）F.听力摘要
9．您认为这个题型对该等程度的学生的难易度如何？
(11%)A. 很难(40%)B.难
(49%)C.一般(0%) D.容易
10．您认为此次整个考试的时间如何？
(0%)A.很长 (6%)B.较长 (82%)C.适中
(12%)D.较短 (0%)E.很短
11．您认为这个题型的评分标准描述如何？
(3%)A.很清楚 (38%) B.較清楚 （59%）C.一般（0%）
D.不够清楚 （0%）E.很不清楚。
Key Words: listening-for-summary, listening comprehension, test format, PRETCO
1．Introduction
The objectives for higher vocational and technical education are to train talented technicians and staffers in the front line of production, service, management, etc. The students graduating from those colleges will go to work immediately. They may use English to take a message over the telephone, take part in talks, meetings or lectures and then write a report, etc. These activities have integrated two or more of the four skills, that is, listening, speaking, reading and writing. However, many graduates are still unable to understand lengthy talks in English on their working sites in spite of 6 or 7 years' study of English. Moreover, some of them can not apply what they have learned into practical work, especially when foreign languages are needed.
Evidently, one of the weakest ability components in their use of English in practical situation is that they cannot understand foreigners' lengthy talks so that they cannot make relevant responses.
To that end, we are proposing an item format for testing listening comprehension, namely, listening-for-summary. By this format, listening comprehension and summary writing are integrated because we believe that in many real-life work situations the technicians, mechanics, staffers, managers, etc., are very likely required to give short but clear reports in foreign languages, either spoken or written, to their superiors, or give briefings to their subordinators, after they have attended training, lectures, negotiation, etc. We also believe that such a test format will probably exert positive effect on the relevant teaching. Therefore, we have conducted a preliminary study on this format for the possibility of using it in the Practical English Test for Colleges (PRETCO).
2. Test Specifications of the PRETCO-LS
A test's specifications provide the official statement about what the test tests and how it tests it. In the following part, only the items which are different from the current listening comprehension part in PRETCO are mentioned.
2.1 Test operations
A summary table of the operations that the PRETCO-LS intends to test is below:
A Explicit meaning comprehension
01 distinguishing main points from supporting details
02 listening for the explicitly stated main ideas
B Implied meaning comprehension
03 synthesizing the input
C Contributory meaning comprehension
04 recognizing phonological features
05 recognizing structural and grammatical features
D Enacting skills
06 reducing what is heard to an outline of main points and important specifics
2.2 Performance conditions
Textual conditions
Texts of the PRETCO-LS will mainly be selected from subject-matter materials which relate to business environment but no specialty knowledge is required. All vocabulary in the test is included in the syllabus. Core vocabulary will be adopted and no technical words. Considering this kind of listening comprehension test format, it is reasonable to choose lengthy materials. Nevertheless, choosing too long materials is also unsatisfactory. One obvious reason is listener fatigue; another is the effect of memory load. So a discourse of about 200-300 words in length is believed to be appropriate.
Task conditions
The format is a listening for summary writing. A selected essay or conversation of about 200-300 words is given. After listening to the essay or conversation, testees are required to write a summary of about one third or fourth of the length of the original material. The material is repeated three times. The text lasts three to four minutes. The total time available for the test is about 15 minutes, including the time to read instructions, deliver the texts and a pause for completing task. Rubrics in English will be presented both aurally on the tape and in written on testees' paper. Testees will be reminded to listen and read the rubrics carefully before performing the task. The speech rate of the test is about 110 wpm, which is slightly slower than 120 wpm stated in the syllabus. The obvious reason is that the testees are not familiar with this new listening comprehension testing format. There is no background noise in the input material of the PRETCO-LS.
Marking system
A.The PRETCO is to test whether testees have reached the requirements stated in the syllabus of the PRETCO.
B.The PRETCO-LS adopts holistic and analytic scoring. First, the markers give reward scores according to the general impression of testees' responses; secondly, the markers give reward scores according to the key points of the summary, which have been clearly specified in the marking scheme. (See Table 1.)
C.Each summary is scored by two markers. Where discrepancies are small, the two scores can be averaged; where they are larger, a third marker will be called in and then the three scores are averaged for the final.
D. Scoring is based on the accuracy and completeness of the information explicitly stated in the testees' summaries. It is not based on mechanics or writing style.
E. It is probably worth mentioning again that in the scoring there is no reason to deduct points for errors of grammar or spelling provided testees can express what they want to express.
3. Pilot test
3.1 Sample
The test was conducted in Shanghai College of Electricity & Machinery Technology. A total of 153 second-year students attended the test. They were selected from a wide range of subject areas including technology/engineering, information/science, economy/management, machinery. All of them took the PRETCO in June 2004. Their scores of the PRETCO were obtained from the test committee so that the correlation studies between the PRETCO-LS and the PRETCO-LC could be conducted. 13 students' scores of the PRETCO were unfit and therefore were discarded. So the actual size of the testee sample for the main trial is 140.
3.2 Administration
The test was administered in five language labs in Shanghai College of Electricity & Machinery Technology. In each lab, two invigilators were assigned to prevent the act of cheating.
4. Data analysis
4.1 Quantitative data analysis
4.1.1 Data descript_ion
Data analysis was made on the PRETCO-Listening Comprehension (PRETCO-LC) and the PRETCO-Listening for Summary (PRETCO-LS) scores of the 140 testees.
The most interesting finding from the descript_ive analysis (see Table 2) is that the standard deviation of the PRETCO-LS scores is wider than that of PRETCO-LC. This result confirms us in the belief that the PRETCO-LS can discriminate testees better than PRETCO-LC does.
In addition, we can find the mean of the PRETCO-LS is lower than that of PRETCO-LC. This difference may be resulted from the fact that most of testees have been used to the Multiple Choice Question (MCQ) format that is employed in the first two sections of the listening comprehension part in PRETCO-LC and are not familiar with the new format used in the PRETCO-LS. Another reason is probably due to the fact that it is comprehensible. In PRETCO-LC, the first two sections belong to MCQs, students can guess the correct answers although they do not understand what they have heard. Moreover, the four choices within each question can give students some clues so that it can help them understand what is going on. Most of testees got most of marks or full marks in first two sections while lost marks in section 3 which required the productive ability. While in the PRETCO-LS, it focuses on "listening and writing" operations, which, to a larger degree, integrate knowledge about vocabulary, grammar, structure and skills in organizing language into listening comprehension, so it is natural for this test to reflect a comprehensive language competence to some extent.
As we refer to the two histograms of the PRETCO-LS and the PRETCO-LC score (see Figure 1), we can see both results fall in a normal curve, which shows that the sample has very good representativeness.
Figure 1 The PRETCO-LC and the PRETCO-LS histograms
4.1.2 Correlation between PRETCO-LS and PRETCO-LC
After examining the results of the 140 testees, we found 9 sets of scores are obviously abnormal. The 9 testees' scores in the PRETCO-LS are much lower than their scores in the PRETCO-LC, with the differences ranging from 7 to 10. Therefore the scores they obtained in the PRETCO-LS are not credible for the data analysis. It is reasonable to discard them. Then correlation studies were conducted based on the valid samples (131 students). The correlation coefficient between the two tests is .635 which is significant at the 0.01 level (see Table 3)
**Correlation is significant at the 0.01 level (2-tailed).
The relatively high correlation between the PRETCO-LS and the PRETCO-LC gives us some confidence to say that, to a large extent, what the PRETCO-LS tests is measuring of listening competence. However, the correlation is not so high as to render the two tests identical in their focuses of testing.
4.2 Qualitative data analysis
Along with the main trial of the PRETCO-LS, a questionnaire survey was conducted to students who took the test. In addition, another questionnaire survey was conducted among 43 teachers coming from the Foreign Language Department of Shanghai College of Electricity & Machinery Technology and all of them have rich experience in training students for sitting for the PRETCO. It should be pointed out that most questions in the questionnaire allow one answer only, but some allow more than one answer. Moreover, some respondents leave questions unanswered occasionally.
4.2.1 Questionnaire to testees
The aim of the questionnaire is to obtain the testees' feedback to the following information:
●Their general attitudes to the PRETCO-LS and the PRETCO-LC
●Their attitudes to the formats
●Whether the timing was sufficient for completing the task
●Whether the delivery rate is appropriate
●Whether the length of the text is appropriate
●Their attitudes to the usefulness of the test for their future job
●The difficulties encountered in comprehending the input
The feedback from the questionnaire is desirable on the whole. 72% of the testees believe the PRETCO-LS is a good measurement of their competence in listening comprehension. Majority of the testees agree that the test conditions such as speech rate and test time are appropriate, and the test can measure the ability to understand spoken English in their future job. Concerning the test formats used in the PRETCO-LS and the PRETCO-LC, the results show that the short dialogue multiple choice format is preferred by most students. Though 89% of the testees rated the format used in the PRETCO-LS as the least preferred, the result of the correlation study shows that this test has a medium-high correlation with the PRETCO-LC. The probable reason for the testees' negative feedback on this format is that the testees were not confident in their performance of the task because the task requires the testees not only to understand the spoken discourse but also to extract the main points and organize them into a summary. Many testees feel such a test task is very demanding. As for the problem they encountered in the PRETCO-LS, distinguishing main points from the details and organizing the summary were rated the most difficult by the testees. This finding proves that the score of the PRETCO-LS can be used to make decision on whether a testee is good or not at listening comprehension.
4.2.2 Questionnaire to teachers
The teachers' questionnaire has many questions in common with those in the students' questionnaire. Additionally, this survey provides us with some other information.
●Whether some reforms should be taken for the PRETCO-LC format
●Their attitudes to the overall design of the test paper
●Their attitudes to the difficulty of the PRETCO-LS
●Whether the marking scheme is reasonable
In general all the teachers gave a very positive response on the overall validity of the PRETCO-LS and the PRETCO-LC. 81% of the teachers think it is necessary to reform the listening comprehension test format of the PRETCO. When asked about their evaluation of the overall design of the PRETCO-LS, 85% of the teachers consider it good or fairly good. Interestingly, teachers' opinions differ greatly from those of the students in the preference of test formats. The “short-dialogue multiple choice” task is the most unpopular with them, followed by the “conversation comprehension multiple choice” task. “compound dictation”,“passage listening comprehension” “short answer question”and “listening for summary” tasks gain most support from these teachers. From the view of teachers, the subjective formats encourage the use of various types of listening comprehension operations, hence resulting in a positive washback effect on teaching. 88% of the teachers are satisfied with the marking scheme of the test.
5. Conclusions and improvements
To sum up, PRETCO-LS has achieved a high validity and reliability with a fairly high correlation with the PRETCO-LC. The correlation shows that the same construct is being tested by the two listening tests. Of course, the correlation coefficient is not higher than .70, which is actually not desirable in this situation because we believe the two formats should not be identical to each other and some new tasks and new operations are introduced. Moreover, the format gains general credit from both teachers and students. It is believed that the test is focusing on the measurement of listening competence rather than any other language traits.
Though our research has produced a preliminary framework for designing an alternative listening test format for the PRETCO, we believe that due to the restrictions in time, human resources, finance, and the researcher's limit capacity of the knowledge of language testing, there must remain much room for further improvement and that there are many issues waiting for other researchers to deal with.
References
[1]Baker, D. 1989. Language Testing. Edward Arnold Ltd.
[2]Buck, G. 2001. Assessing Listening. Cambridge: Cambridge University Press.
[3]Carroll, B.J.1980. Communicative Performance Testing. Pergamon Press Ltd.
[4]Carroll, J. B. & Hall, P. 1985. Write Your Own Language Tests. Oxford: Pergamon Institute of English.
[5]刘鸿章，1997，《重视实用能力的考核》， 《外语界》， 第四期。
[6]金艳，2002，《高级英语阅读测试的开发和效度研究》，上海：上海交通大学出版社。
[7]教育部高等教育司，2000， 《高职高专教育英语课程教学基本要求》（试行），北京：高等教育出版社。
虞苏美，杨乾龙，2001，《商务英语听力》，北京：高等教育出版社。
Appendix 1
考生问卷调查
1．你认为目前的PRETCO听力测试能在什么程度上客观，真实地反映你的英语听力水平？
(12%)A很大程度 (58%) B一般程度 (20%)C 较小程度 (0%)D不能
2．你认为目前的PRETCO听力测试语速是否合适？
(29%)A太快(64%) B 正好 (7%)C太慢
3．你认为目前的PRETCO听力测试答题时间是否合适？
(16%)A 太长 (65%)B正好 （19%）C太短
4．你认为刚才完成的听力考试在什么程度上真实地反映了你的英语听力水平？
(24%)A 很大程度(48%)B一般程度 (28%)C较小程度(0%)D不能
5．你认为刚才完成的听力考试的语速是否合适？
(24%)A 太快(71%)B正好 (5%)C太慢
6．您认为刚才完成的听力考试中文章的长度是否合适？
（3%）A 太长（14%）B 长 （73%）C正好 （8%）D 短（2%）E 太短
7．你较喜欢的听力题题型为：（可多选）___________
(92%)A 短对话多项选择题
(45%)B 长对话多项选择题
(12%)C短文理解多项选择题
(5%)D复合式听写
(3%)E 间答题
(13%)F 听力摘要（Listening for Summary）
8．你不喜欢的听力题题型为：（可多选）___________
(3%)A 短对话多项选择题
(56%)B 长对话多项选择题
(88%)C短文理解多项选择题
(95%)D复合式听写
(93%)E 间答题
(89%)F 听力摘要（Listening for Summary）
9．你认为完成刚才的听力考试，时间是否合适？
(1%)A很长 (13%)B较长(74%)C. 适中(12%)D. 较短(0%)E. 很短
10．你对这种听力题型熟悉吗？
(2%)A 很熟悉 (33%)B较熟悉 (41%)C听说过 （24%）D从来没有听说过
11．你觉得这种题型对你将来的工作会不会有用？
（17%）A很有用（72%）B可能有用 (11%)C 不会有用
12.在本次考试听力过程中，你遇到哪些困难？(可多选)
(14%)A. 语速太快 (36%)E无法把握重点
(55%)B词汇太难 (87%)F听懂但不会组织语言
(65%)C句子结构复杂 G其他___________
(42%)D题型不熟悉
Appendix 2
教师问卷调查
1．您认为目前的PRETCO 的听力测试能在什么程度上真实地反映考生的英语听力水平？
（28%）A.很大程度 （68%）B.一般程度
（4%）C.较小程度（0%）D.不能
2．您对目前的PRETCO听力考试内容题材是否喜欢？
（12%）A. 很喜欢（45%）B.较喜欢
（32%）C.一般 （1%） D.不喜欢
3．您对目前的PRETCO听力考试的题型是否喜欢？
（5%）A. 很喜欢（32%）B.较喜欢
（63%）C.一般(0%)D.不喜欢
4．您认为是否有必要改革现在的听力题型？
(24%)A. 很有必要 (57%)B.较有必要
(19%)C.不必要
5．您认为本次考试(The PRETCO-LS)能在什么程度上反映学生的英语听力水平？
(23%)A.很大程度(69%)B.一般程度
(8%)C.较小程度(0%)D.不能
6. 您对此次听力考试试卷的总体设计评价如何？
（5%）A很好(67%)B 较好
（28%）C一般（0%） D不好
7．您较喜欢的听力题型是 (可多选)___________
(6%)A. 短对话选择题（82%）D. 复合式听写
(21%)B. 长对话选择题 （79%）E. 间答题
(58%)C. 短文理解选择题 （76%）F.听力摘要
8．您不喜欢的听力题型是（可多选）___________
（83%）A. 短对话选择题（2%）D. 复合式听写
（45%）B. 长对话选择题（5%）E.间答题
（11%）C. 短文理解选择题（7%）F.听力摘要
9．您认为这个题型对该等程度的学生的难易度如何？
(11%)A. 很难(40%)B.难
(49%)C.一般(0%) D.容易
10．您认为此次整个考试的时间如何？
(0%)A.很长 (6%)B.较长 (82%)C.适中
(12%)D.较短 (0%)E.很短
11．您认为这个题型的评分标准描述如何？
(3%)A.很清楚 (38%) B.較清楚 （59%）C.一般（0%）
D.不够清楚 （0%）E.很不清楚。