Assessment questions - a guideline

Let me continue with my earlier post on assessment guidelines. The following is an excerpt from my team Wiki (that runs off my PC). Of course these guidelines apply to test technical subjects, however, many of these points may be applicable to create assessment questions for other subjects too.

1. Questions should use positive connotation only.
   Example: Which one of the following is a feature of Logo?
   Non-example: Which one of the following is not a feature of Logo?
2. Answer options too should use positive connotation.
   Example: Raju is yet to save the file.
   Non-example: Raju did not save the file.
3. It is advisable to restrict types of questions to MCQs so that the lot can be used for online tests, quickly, if required.
4. Questions stem should be articulate and should not be convoluted.
   Example: How will you copy-protect a CD?
   Non Example: How will you ensure that contents on a CD, either applications or documents, are usable to general public but not available for illegal copying?
5. While referring to tools on the toolbar, pictures of the tools should be shown instead of referring to them by their names. This applies to both the question stem and the answer options.
6. Questions should not be repeated across sets (versions) of question papers, if the sets are prepared as a printable version.
7. Questions should test the skill level of the learner to perform a task and not his/her ability to remember names of buttons, tabs and tools or identify the location of a tool on the screen/menu.
8. The distracters (non answers) should not be non existent options. For example, an option should not be a menu or a command that is either non-existent or wrongly associated with a location. The user is always guided through the GUI components and thus does not require to memorise the menu sequences.

... is what I call tacit knowledge

Assessment Blueprint

Starting to write some questions for your learning module? Start with a good blueprint to give you a complete picture of what, how much and which level of learning would you address through your questions.

An assessment blueprint provides an exhaustive list of objectives relating to each chapter or learning module. The following are the features of a blueprint.

1. A list of objectives mapping to each chapter or module (for example, Word, Excel, and Power Point in total) is prepared.
2. Each objective is identified with a corresponding level in the Blooms Taxonomy. Check with your client (if you find them knowledgeable) if they are specific about the level to be addressed.
3. Each objective should test the learner on only one specific task. Example: To compare the menu commands to publish a Web page. Non-example: To create, save, print and modify a workbook.
4. Objectives should address the skill level of the learner to perform a task and not his/her ability to remember names of buttons, tabs and tools or identify the location of a tool on the screen/menu.

Such a blueprint can be used to prepare 3 or more sets of assessment question papers. Using a standard blueprint to prepare multiple sets will ensure that all learners are tested on the same set of objectives irrespective of the question paper set they attempt in their exam.

More on guidelines to write questions to come...

Knowledge Management - key to trap tacit knowledge

Life is a constant cycle of learning. I'm not being philosophical, just beginning my write up with something higher in the order of thinking. To me, knowledge can be acquired from reading books and through practice. Knowledge available in books could be so specific to a given situation that it may not apply to other situations, or too generic that you have to rely on your capacity to use it to your best.

Tacit Knowledge
Though theories and principles cover many aspects in a domain, knowledge gained through experience often go undocumented. Loss of such invaluable cognition is one of the key issues that many organisations fight to contain.

Tools to Tackle the Trouble
I would like to share what I have been experimenting for quite some time now to document tacit knowledge at work - Wiking and Blogging. More than an year back, I downloaded an intranet version of Wiki (quick web) and installed it on my computer (I have IIS running on my system, serving as an intranet site on my network segment)

I had to slowly drive the point of using the Wiki (among my writers and editors) to document their learning at work. Now I see that I have a database of checklists, case studies and thumb rules that people have derived through the Wiki over a period. I see that its growing to become a powerful tool to trap tacit knowledge which is sharable within a group of knowledge workers.

I took to blogging quite late. I have sincerely tried to blog whatever ideas I conceived so far and maintain a record of some tacit knowledge. I see that my organisation ( at least my division) has also recognised the power of blogs and wikis that I see some initiative on setting up secured Wiki for a few management discussions.

I see blogs and wikis as effective tools for knowledge management. I shall continue blogging and wiking...

Knowledge Models - tools to drive home the concepts

Knowledge Models help drive home any concept effectively. Diagrammatic representation of theories through Knowledge Models, such as concept maps, decision ladders, and process maps enable easy understanding and retention.

I hit upon the following Web site on the knowledge models, which also provides some good examples to understand and use the models better. Nice read.

http://www.epistemics.co.uk/Notes/90-0-0.htm

Constructivism impacts learning

The subtle existence of Constructivism can be found in almost all forms of learning materials these days. Strategising the learning technique for students to build upon their prior observations/knowledge is the key to effective learning.

Recently I read a good article on Constructivism and its impact on curriculum, instruction and assessment. Thought I would leave a link to the same here.
http://www.funderstanding.com/constructivism.cfm

At Work 

Churning IDs - Part 2

So how to churn IDs? - I have seen a better way to churn IDs for the work on hand. This is effective yet requires some time and energy.

This is the reverse engineering of what I had written in my part1 of this message (This message is a sequel to the one I wrote a few days back).

I cannot explain it further coz it would invite political trouble :-)

Churning IDs - Part 1

With the advent of e-learning, instructional designers are the most-sort-after species these days. I have personally observed that getting a bunch of creative minds to work on instructional designing is not duck soup. IT involves,
1. Identifying the profile of candidates to suite the project (IT, non IT, special subjects...).
2. Sourcing prospective candidates from some consultant or through WOM or ad.
3. Interviewing, testing, negotiating...

After all this, find that the one you chose is not the one you wanted!

How to solve this problem?
Fix. Take in people the way you have planned the recruitment process, then put them through an intensive training on instructional strategies, theories, methodologies, style guides, tests... Shape them up to take on the challenge, give them that float board to get them into water.

Solution. This would take a minimum of one month, but sure to work. This has worked in the past but let me warn you. You need some brand mileage to take up this solution.

But wait... I reserve that solution for my next post.