TUTORIAL EVALUATION CRITERIA
Tutorial Evaluation Criteria
The evaluation criteria for tutorials are grouped into five groups, as defined below:
1. Topic The subject matter of the tutorial is clearly defined The proposed scope is appropriate to the duration of the tutorial (half or full day) The topic’s position within the framework of systems engineering is clearly defined The intended audience is defined
2. Value Conference participants can put the topic to practical use Use of the topic is likely to result in significant improvements in a participant's organization The topic will provide professional and / or personal value to conference participants The topic is in high demand by systems engineers This is a topic that would attract conference participants
3. Appropriateness The topic supports INCOSE SE Principles and the SE Body of Knowledge The topic provides attendees with motivation for understanding SE applications The material is educational in nature, and does not represent company products
4. Material The material presented in the proposal addresses the topic adequately The structure and format of the material is good. The approach is didactically sound The presumed knowledge level reflected by the material correlates with the intended audience
5. The Presenter (if more than one, these criteria apply to each one) Has adequate knowledge of and experience in the area of systems engineering involved Has a background in teaching or training Gives good oral presentations (if known by the reviewer) Has the visibility and stature within INCOSE to attract attendees
Consequently, besides the general requirements that the tutorial shall not be used for the promotion of any commercial product or interest, and that the tutorial shall not be used to promote or voice an opinion on political or religious matters, the evaluation criteria for tutorials address all five of the above components.
Insight into each of the Evaluation Criteria is presented in the following sections to assist the author(s)
/ tutorial presenter(s) in achieving high quality of the submitted proposal, thus increasing the likelihood of acceptance.
Procedure for the reviewers:
a. Read or refresh the reading of the evaluation criteria to ensure an understanding of what to review against. b. Access the Tutorial Review Database, and bring up one of the assigned tutorials on the screen, or download and print.
2011
c. Read the proposal through quickly to get an understanding of its objective and structure. A good proposal will enable the reviewer to easily comprehend the scope and intended outcome of the tutorial. d. Re-read more thoroughly, jotting down comments.
e. Grade each of the five groups of criteria using the following scale:
1 = Poor, 2 = Fair, 3 = Good, 4 = Very Good, and 5 = Exceptional.
f. Assess overall contribution of the proposal to the symposium and select one of the following recommendations: Accept as is, Accept with modifications, or Reject; the latter two recommendations require the reviewer to enter a corresponding justification. In carrying out this assessment, answering the following questions might assist: Would you attend this tutorial if it were presented at the Symposium? What level of interest do you believe others within the systems engineering community would have in this tutorial? Compared with other tutorials you have attended and/or reviewed, how does this proposal rank? g. Provide constructive comments. Limit comments to what it would take to make the tutorial a better tutorial. Opinions and conclusions of the reviewer, if a disagreement, should not be captured, unless made as a constructive “have you considered” statement. Comments might include examples such as “the section on xyz should be shortened” or “the section on abc should be expanded to include a more detailed explanation and rearrangement. I suggest…”. Be specific.
Words of encouragement such as, “This is a great proposal and the tutorial should be of great value to participants” are always welcome when deserved.
Note:
It is not prudent to rate a proposal highly if the submitted material is incomplete; it should be a draft that clearly indicates the finished product.