Assessment

From e-Learning Documentation

Revision as of 15:06, 22 October 2010 (view source) Jolaine (Talk | contribs) (→Create a Survey in Assessments) ← Older edit		Revision as of 15:08, 22 October 2010 (view source) Jolaine (Talk | contribs) (→Create a Survey in Assessments) Newer edit →
Line 336:		Line 336:
	Creating a Survey is exactly like [[Assessment#Create an Assessment in Assessments |creating an assessment]] except that you choose "Survey" under "Type."		Creating a Survey is exactly like [[Assessment#Create an Assessment in Assessments |creating an assessment]] except that you choose "Survey" under "Type."
-	Managing Survey submissions is exactly like	+	Managing Survey submissions is exactly like [[Assessment#Manage Student Submissions in Assessments | managing assessment submissions]].

---

Revision as of 15:08, 22 October 2010

Contents 1 Overview 2 Create and Manage Question Pools in Assessments 2.1 Create a Question Pool in Assessments 2.2 Add Questions to a Question Pool in Assessments 2.3 Import a Question Pool in Assessments 2.3.1 Remove, Copy and Move Questions Within Question Pools in Assessments 2.3.1.1 Remove Questions from Question Pools Assessments 2.3.1.2 Copy Questions from one Question Pool to Another in Assessments 2.3.1.3 Move Questions from one Question Pool to Another in Assessments 3 Question Types in Assessments 3.1 Multiple Choice 3.1.1 True/False 3.1.2 Essay 3.1.3 Fill In The Blank 3.1.4 Matching 3.1.5 Likert Scale 3.1.6 Task 4 Create an Assessment in Assessments 4.1 Add a Part in Assessments 5 Publish an Assessment in Assessments 5.1 Settings in Assessments 5.2 Grant Special Access in Assessments 5.2.1 Edit Special Access 5.2.2 Delete Special Access 6 Manage Assessments in Assessments 6.1 Unpublish an Assessment in Assessments 6.2 Delete an Assessment in Assessments 6.3 Archive an Assessment in Assessments 6.3.1 Restore an Archived Assessment in Assessments 7 Manage Student Submissions in Assessments 7.1 View and Edit Student Submissions in Assessments 7.1.1 Grade Anonymously 7.1.2 Grade by Question 7.1.3 View Summary of Data 8 Create a Survey in Assessments

Overview

There are two online testing tools in Sakai, Assessments and Tests & Quizzes. You may choose whichever of these tools best meets your needs. In general, Tests & Quizzes offers more function than Assessments, but many people find the Assessments tool more intuitive and easier to use.

The two tools are very different in their approach to online testing.

Assessments REQUIRES the use of Question Pools and those Question Pools MUST be structured in a specific way. Question Pools must contain questions of the same point value and difficulty level. This is a good practice and a helpful way to set up your Question Pools, resulting in a very nicely organized collection of Question Pools. Assessments also has an easy process for assigning an assessment to special needs students that need added time, but there is no option to assign an assessment to a group.

Assessments also offers the option to require students to acknowledge the Honor Pledge before they can submit the assessment. Certain question types, Multiple Choice, True/False, and Essay, can be imported into Assessment from Respondus. Question Pools can be imported from Tests & Quizzes into Assessments.

Probably the biggest limitation of Assessments is that once an assessment has been accessed, even by a Test Drive, that Assessment cannot be edited or deleted. It can only be archived.

Tests & Quizzes offers Question Pools, but does not require their use. You may enter questions directly into an assessment without those questions being assigned to a Question Pool. You can have questions of varied difficulty levels and point values in one Question Pool. You can create Sub-Pools within Question Pools to further aid organization. Question Pools in Tests & Quizzes are associated with the creator's Username, not the course in which the pools were created. So if you are instructor for ABC1234 and CBA4321 and you create five Question Pools in ABC1234, you will also see those five pools when you go to your CBA4321 course. This is helpful in that you always have access to your Question Pools, but it requires you to pay attention to naming conventions. You would not want to name a Question Pool "Exam 1."

Tests & Quizzes offers the added security of limiting an assessment to a specific IP address, or group of IP addresses. You have the ability to show students the statistics on an assessment and individual questions within the assessment. Questions can only be entered manually, or by cutting and pasting a properly formatted text document. An assessment in Tests & Quizzes can be assigned to a group. You can edit many aspects of a published assessment, including the ability to edit a question and regrade the assessment.

Create and Manage Question Pools in Assessments

As noted above, Question Pools are required in Assessments. You must create Question Pools before you may create an assessment.

Create a Question Pool in Assessments

From within the course where you want to create a Question Pool:

1. Click Assessments in the left menu bar.
2. At the top of the Assessments page, click Question Pools
3. Click Add Pool
4. "Pool Title" - enter a descriptive title for this Question Pool
5. "Description" - you may enter a description of this pool if desired. The Content Editor is available if you want to format this text.
6. "Point Value" - you must enter a Point Value that will be applied to every question in this Question Pool. It is not possible to have questions with different point values in the same Question Pool.
7. "Difficulty" - select the difficulty level of the questions in this Question Pool. It is not possible to hvae questions with different difficulty levels in the same Question Pool.
8. Click Done

Add Questions to a Question Pool in Assessments

When the Question Pool is created, you are taken to a screen that allows questions to be added. If you want to add questions to a Question Pool that was previously created, access this screen by clicking Assessments > Question Pools > Hyperlinked Name of Question Pool.

1. Click Add Question
2. Select Question Type
3. Click OK
   • Each question type provides instructions for creating that type of question on the question creation screen.
4. Click Done

Import a Question Pool in Assessments

You may import a properly formatted text document of questions. This process creates a new Question Pool.

1. Using Wordpad or other plain text editor, enter tests questions using the proper format:
2. From within the Question Pool screen (Assessments > Question Pools) click Import
3. Select "Paste Questions & Import"
4. Copy the text from your document and paste it in the box provided in Sakai. Make sure that there is a space between each question.
5. Click OK

If the import was successful a new question pool was created. The newly created Question Pool will be untitled and not have points associated with it. The difficulty level will be at the default, "Level 3." To edit the Question Pool details, click on the "Edit Tool" icon beside the name of the Question Pool. On this screen you are able to rename the Question Pool, provide a Description if desired, enter the Point Value and adjust the Difficulty Level.

Remove, Copy and Move Questions Within Question Pools in Assessments

Remove Questions from Question Pools Assessments

1. Open the Question Pool (Assessments > Question Pools > Click the hyperlinked name of the Question Pool) that contains the question(s) you want to Remove (delete).
2. Check the box next to the name of the question(s) you want to remove.
3. Click Delete
4. In response to the "Are you sure you want to delete the selected questions?" click Delete.

Copy Questions from one Question Pool to Another in Assessments

If you copy a question to a different pool, the question will then appear in both question pools.

1. Open the Question Pool (Assessments > Question Pools > Click the hyperlinked name of the Question Pool) that contains the question(s) you want to copy to a different pool.
2. Check the box next to the name of the question(s) you want to copy.
3. Click Copy to Pool
4. A screen is opened showing all available pools. Select the pool to which you want to copy the question.
5. Click OK.

Move Questions from one Question Pool to Another in Assessments

If you move a question to a different pool, the question will be removed from the original pool and placed in the new pool.

1. Open the Question Pool (Assessments > Question Pools > Click the hyperlinked name of the Question Pool) that contains the question(s) you want to move to a different pool.
2. Check the box next to the name of the question(s) you want to move.
3. Click Move to Pool
4. A screen is opened showing all available pools. Select the pool to which you want to move the question.
5. Click OK

Question Types in Assessments

There are several types of questions available in Assessments. When you create each of these question types, you will be given instructions on how to compose the question if it is not obvious. Below is a brief description of each type and when you might want to use it.

Multiple Choice

There are two types of Multiple Choice questions:

• Single Correct - there is only one acceptable answer
• Multiple Correct, Multiple Selection - there are several options that are considered "correct" and students may choose more than one.

Multiple Choice questions allow for the answers to be randomized each time the question is displayed to a student so that the correct answer is a different option each time. You can also require the students to provide a rationale for their answer so that you can verify that the student knows the answer and isn't just guessing.

True/False

True/False questions ask the student to determine if a statement is true or false.

Essay

Essay questions provide two ways for students to submit their answers.

• A text box with a Content Editor
• The ability to attach a file to the question

The instructor decides to allow one or both of these options. This question type must be manually graded.

Fill In The Blank

Fill in the Blank questions ask the student to insert a word or words into the question text to complete a sentence or paragraph. While this question type is automatically graded, the student's entry must match exactly the option(s) provided by the instructor. There is some flexibility in providing acceptable answers, matching case, etc., but the best practice is to only use this type of question with totally unambiguous test questions.

Matching

Matching type question require that the student matches one term, phrase or image to another. For example, you could insert images of plants and the student would have to choose the name of the plant from a list provided.

Likert Scale

This question type is only available as a Survey question. The participants answer this question using a scale chosen by the instructor, such as: "On a scale of 1-10, what did you think of . . .?"

Task

This question type is exactly like Essay, except that you have the added option to say "No answer is expected for this question." You would use this option if the student was going to be graded on, for example, an in-class performance. This is comparable to the "non-electronic" submission feature in Assignments. This question type must be manually graded.

Create an Assessment in Assessments

Once you have constructed your Question Pools, you can create Assessments. From within the course where you want to create an Assessment:

1. Click Assessments
2. You are taken to the main Assessments page. Click Add.
3. Select from one of three types of assessments:
   • Test - a tool to evaluate student knowledge.
   • Assignment - this option behaves exactly like a Test, but it gives a different icon beside the name of the assessment in the assessment list.
   • Survey - the survey is intended to gather opinions from anonymous users. There are no correct answers or points awarded.
4. "Title" - enter a descriptive title for this assessment
5. Click Add Instructions if you want to give the students additional instructions. By default, the assessment will provide information such as dates available, time limit, attempts allowed, etc.
6. Click Done

The assessment will have the invalid icon beside it until questions are added.

Add a Part in Assessments

Every assessment must have at least one Part. You may add many Parts, using them to organize the questions in the assessment. You might have a Part that pulls 10 random easy questions from a Question Pool and a Part that pulls 5 random difficult questions from a different Question Pool. You may add specific instructions for each Part.

To add a Part:

1. Click on the hyperlinked name of the Assessment. Under the heading "Add Parts" you have two options:
   • "Add Manual Select" - this option shows all questions from all pools and allows you to check a box to add this question to the Part.
   1. Click Add Manual Select
   2. "Part Title" - enter a descriptive title for this Part.
      1. Click Add Instructions to provide additional instructions to the Part.
   3. Click Select Questions
   4. Check the box beside the question(s) you want to add to this Part.
      1. To make it easier to locate the question(s) you are looking for, you have the ability to sort the list by Pool, Type and Assessment or Test questions. Sort from the drop-down menu beside each option.
   5. Click Done
   6. Click Done
   • "Add Random Draw" - this option allows you to randomly choose a specific number of questions from a Question Pool. If you have a Question Pool that contains 45 questions, you can give each student 5 randomly drawn so each student could get entirely different questions.
   1. Click Add Random Draw
   2. "Part Title" - enter a descriptive title for this Part.
      1. Click Add Instructions to provide additional instructions to the Part.
   3. All Question Pools will be shown, along with the number of questions available (if questions from this pool have already been added to a different part of the assessment, they will not be available for random draw in this Part.), point value and difficulty level. In the "Draw" box, enter the number of questions you want pulled from that Question Pool.
   4. Click Done
   5. Click Done

Publish an Assessment in Assessments

Once you have an assessment with at least one Part and questions, you are ready to determine how you want the assessment to be delivered to the students.

NOTE: Once an assessment has been published, you can no longer add, remove, or change questions. If you find a problem with a question, want to change the order of the parts, etc., you will have to duplicate the assessment and make the changes in the duplicate. If the original assessment has submissions, even a test drive, you cannot delete it. You can only archive the assessment. You probably want to change the name of the original to include "faulty" in the title so you know not to retrieve and use that assessment from the archive at a later date.

Settings in Assessments

To edit the settings and publish as assessment:

1. From the Assessments list on the main page of the Assessments tool, click the Publish icon to the far right of the Title of the assessment you want to publish.
   1. "Title" - you may edit the name of the assessment here.
   2. "Type" - displays the assessment type chosen when the assessment was created. It is possible to change it here, but not recommended if you have already added questions since the various types require unique question types.
   3. "Open" - this is the date and time that the assessment becomes available for students. If you do not enter anything, the assessment will be available as soon as you publish. Click the calendar icon to choose a date and time.
   4. "Due" - this is the date and time after which the assessment is considered "late." Click the calendar icon to choose a date and time.
   5. "Accept Until" - this is the date and time after which the assessment will no longer be available to students. If you make this the same as the "Due" date and time, late submissions will not be accepted.
   6. "Tries" - How many attempts do the students get? Default is that they get one attempt. You may change the number to anything you want, or change the selection to "Unlimited."
   7. "Time Limit" - The default is that there is no time limit. If you want to provide a time limit, change the selection to "Timed" and enter the time. Click Instructions to see the format for entering the time.
   8. Make Review Available
      • "Upon submission" - as soon as students submit the assessment, they can see the review options you have chosen to release.
      • "Upon release" - the instructor controls when the review options are released to students.
      • "After date" - you choose a specific date and time for the release of review options. When you select this option, a calendar icon is provided for your use in choosing the release date and time.
      • "Never" - students do not get to review their assessments.
   9. "Show Answer Key & Items Scores" - when the review is released, what aspects do students get to see?
      • "For all answers" - students see the answer key and score for all answers.
      • "Only for correct answers" - students only see the answer key and score for questions they got correct.
      • "Never" - students do not see answer key and score in the assessment submission.
      • "Show feedback" - default is that students see any feedback you've provided in the questions. Uncheck the box if you do not want feedback revealed.
   10. Release Grades
       • "Automatic release" - by default, the scores are released based upon your choice in the "Make Review Available" option.
       • "Manual release" - if you choose this option, you will release the scores one-at-a-time within each assessment, or choose "Release evaluated submissions" or "Release all submissions" from the submissions page. This option is good for assessments that contain questions that must be manually graded.
2. Send to Gradebook - if you check this box, a column is automatically created in the gradebook.
3. Anonymous Grading - This option hides the username so the assessment can be evaluated without knowing who submitted the assessment. To restore the student names after grading is complete, come back to settings and uncheck this box.
4. Password - if desired, enter a password that students must enter before they will be granted access to the assessment. This option could heighten security for proctored classrooms if the password is not given out until all participants are gathered in the classroom.
5. Honor Pledge - If you check this box, students will be required to check a box acknowledging the honor pledge before they are allowed to submit their assessment.
6. Hints - if you have provided "hints" during the construction of your questions, you may check this box to show the hints during the delivery of the assessment.
7. Navigation
   • "Flexible order" - Students are provided with a table of contents which allows them to answer the questions in any order they wish.
   • "Strict order" - The questions must be answered in the order in which they are delivered. There is no going back to a previous question.
8. Layout
   • "One question per page" - each question is displayed on its own webpage.
   • "One part per page" - each Part is displayed on its own webpage.
   • "All questions on one page" - the entire assessment is displayed on one page.
9. Part Numbering
   • "Continuous" - questions are numbered sequentially, regardless of how many Parts there are.
   • "Restart on each part" - each Part starts numbering at 1.
10. Final Message - Provide a final message to students if desired. You might use this to let them know when scores will be released, what they should do after completing the assessment, etc.
11. Click Publish

Grant Special Access in Assessments

Assessments allows you to add special access for specific students, for example, to give extra time to special needs students, without having to create a separate assessment for them. You can also use this option to allow a different time-frame or number of tries.

1. Click Assessments in the left menu bar
2. Beside the assessment to which you want to add special access, click the "grant special access" icon at the far right of the screen.
3. Click Add
4. The names of all participants appear in the box. Select the name(s) to whom you want to grant special access.
5. Provide the Open, Due and Accept Until dates for the user(s) if they are different from the dates of the main assessment.
6. Adjust the Time Limit if desired.
7. Adjust the number of attempts if desired.
8. Provide a password if desired.
9. Click Done. You will be returned to the "Special Access" list where you see your new access listed.

When you return to the main list of assessments, you will see that the grant special access icon has changed to show that this assessment has special access added.

Edit Special Access

1. Beside the assessment, click the "grant special access" icon.
2. Click the hyperlinked name of the user(s) to whom the access has been granted. The edit screen is opened that allows you to make changes.
3. Click Return

Delete Special Access

1. Beside the assessment, click the "grant special access" icon.
2. Check the box beside the special access you want to delete.
3. At the top of the page click Delete
4. In response to the "Are you sure you want to delete the selected special access?" message, click Delete.
5. Click Done.

Manage Assessments in Assessments

Various icons identify the status of assessments in your list.

• means the assessment has no questions and cannot be published.
• means the assessment is currently available to students.
• means the assessment has submissions that need to be graded.

Unpublish an Assessment in Assessments

You may wish to unpublish an assessment if you decide you do not want it to be available to students. You cannot make changes to the questions or Parts once it is published, but you can change settings. If you unpublish an assessment that has submissions, you will lose access to those submissions until you republish.

To unpublish an assessment:

1. Click Assessments from the left menu bar
2. Check the box beside the assessment(s) you want to unpublish.
3. Click Unpublish at the top of the list.
4. In response to the "Are you sure you want to unpublish the selected assessments?" message, click Unpublish

Delete an Assessment in Assessments

Assessments cannot be deleted if there are any submissions, or even if you have done the "test drive." If the assessment has been accessed, you can only archive to remove it from the list. "Live"

To delete an assessment that has not been accessed:

1. Click Assessments in the left menu tool bar
2. Check the box beside the assessment(s) you want to delete.
3. Click Delete at the top of the list.
4. In response to the "Are you sure you want to delete the selected assessments?" message, click Delete.

Archive an Assessment in Assessments

Once an assessment has submissions, it cannot be deleted, but can be archived. Archived assessments can be retrieved for future use.

To archive an assessment:

1. Click Assessments in the left menu bar
2. Check the box beside the assessment(s) you want to archive.
3. Click Archive at the top of the list.
4. In response to the "Are you sure you want to archive the selected assessments?" message, click Archive.

Restore an Archived Assessment in Assessments

To restore an archived assessment:

1. Click Assessments in the left menu bar
2. Click Restore at the top of the list.
3. All archived assessments are listed. Check the box beside the assessment(s) you want to restore.
4. Click OK

The assessment is now in the list of available assessment.

Manage Student Submissions in Assessments

Once students begin to take the assessments, you will be able to view their submissions, make edits, comments and grade those questions that require manual grading.

View and Edit Student Submissions in Assessments

To view the student submissions:

1. Click Assessments in the left menu bar.
2. Click Grading at the top of the page.
3. Click the hyperlinked name of the assessment you want to grade.
4. The "Grade Submissions" screen is opened. You will see a list of all students who have access to the assessment, the number of attempts, date and time the assessment was submitted, the auto score and a box for the Final score (empty if there are questions that must be graded manually). There is also a checkbox that shows if the submission has been evaluated and/or released at this point, plus the icon to give special access to this student. Click the hyperlinked name of the student whose submission you want to view.
5. On this screen, you see each question, the student's response, the correct response, the points awarded. You may make adjustments to points, provide comments on each question and/or comments on the overall submission.
6. Check the "Mark as evaluated box if desired. This serves to let you and other graders know that this submission has already been evaluated.
7. If grades are not already released and you are ready to release this student's grade, check the box labeled "Release grade to submitter."
8. Click Done, or Prev to see the previous submission in the list, or Next to see the next submission in the list.

Grade Anonymously

If you want to grade student scores without knowing who submitted, you will need to set the assessment to anonymous grading.

1. Click Assessments in the left menu bar.
2. Beside the assessment, click the icon.
3. Under the heading "Anonymous Grading," Check the box beside "Suppress username while grading."
4. Click Done.
5. You are returned to the list of assessments. At the top of the screen, click Grading.
6. Click the hyperlinked name of the assessment.
7. Under the "Name" column, you will only see "User" for each submission. Click User to evaluate as above.
8. Once you have completed grading, you can go back to settings and uncheck the "Suppress username while grading" box to reveal the student names.

Grade by Question

Rather than grade an entire submission, you may grade by a particular question. This is a good option if you have a faulty question and want to go in and give everyone credit that missed it due to the error.

1. Click Assessments in the left menu bar.
2. Click Grading at the top of the screen.
3. Click the hyperlinked name of the assessment you want to grade.
4. Click Grade Questions at the top of the list.
5. All the questions that were delivered by this assessment are listed. Click the name of the question you want to grade.
6. You will see the question and then a list of all the students who received that question and can edit and make comments on each submission. Click Done.

View Summary of Data

Summary of Data provides statistics on each question, showing how many students chose each answer, and what percentage got the answer correct. To view the Summary of Data:

1. Click Assessments in the left menu bar.
2. Click Grading at the top of the screen.
3. Click the hyperlinked name of the assessment you want to grade.
4. Click Summary of Data at the top of the list.

Create a Survey in Assessments

A Survey can be created that will gather anonymous opinions from participants. Create a Question Pool that contains your Survey Questions before you create the Survey. Survey questions can be created using any question type. As you create each question, you can check the box which designates this question as a survey question, which will remove the option to designate a correct answer. Once you have your questions in a pool, you may create the Survey.

Creating a Survey is exactly like creating an assessment except that you choose "Survey" under "Type."

Managing Survey submissions is exactly like managing assessment submissions.