Question Guidelines for SAP Certification

Item writing guidelines Manual Version 2 April 2014

Table of Contents Introduction .............................................................................................................................................................. 1 General guidelines .................................................................................................................................................... 2

Certification levels ................................................................................................................................................. 2 Target audiences for SAP exams ......................................................................................................................... 3 Relevance ............................................................................................................................................................ 4 Conciseness ......................................................................................................................................................... 6 Language ............................................................................................................................................................. 6

Question types .......................................................................................................................................................... 7 Multiple choice ...................................................................................................................................................... 9 Multiple response ................................................................................................................................................ 10

Item writing process ............................................................................................................................................... 11 Item writing rules .................................................................................................................................................... 13

Overview ............................................................................................................................................................ 14 Rules .................................................................................................................................................................. 16

Examples ................................................................................................................................................................ 19 Pronouns ............................................................................................................................................................ 19 Single or multiple sentence stems ....................................................................................................................... 21 Answers .............................................................................................................................................................. 23

Terminology ............................................................................................................................................................ 24 Exercise .................................................................................................................................................................. 25

SAP Item writing guidelines 1

Introduction Writing test items for a certification exam is a creative process. Item writing rules help in this process but they are not a replacement for your experience and creativity. We cannot guarantee that you will always write good items if you follow the rules. However, we can guarantee that you will not write good items if you ignore the rules.

The plan We start by describing the general guidelines for item writing. We explain the basic item types that you can use to formulate your questions (multiple choice and multiple response). We continue with a description of the item writing process and present the core item writing rules. We conclude with some examples of good and bad items.

Why should you be interested in following the rules? Quite simply: if you follow the rules, your items have a good chance of being accepted. The more items we can accept, the closer you get to Bronze, Silver, or Gold status. (See the SAP Community Network for more details on the rewards.)

A few words of caution There is nothing creative in copying test questions from existing material. If you remember a test item from a certification exam you passed (or not), then do not submit this item. If you remember a test item from training material, then do not submit it. If you plan on submitting a publicly available (sample) question, dont do it. We already know these questions. We will not accept them. You will not receive credit for them and will just be wasting your time.


General guidelines You must have passed at least one SAP certification exam. This means you are already familiar with SAP exam questions. We will try to guide you through the process of writing them.

Certification levels SAP currently offers three levels of certifications: Associate, Professional and Specialist. SAP certification exams are typically based on job and task analyses that help to distill the important knowledge and competencies an exam will assess.

Associate Associate exams test broad, fundamental knowledge and competencies in an SAP solution or methodology. Associate exam candidates may have project experience in the SAP solution for which they want to be certified. Normally this is not a requirement unless stated in the exam description.

Professional Professional exams usually target consultants with four or more years of project experience in the related SAP solution.

Specialist Specialist exams target consultants who have gained an Associate certification in a particular solution and chosen to validate their specialist knowledge in an associated area.

Consequence for your contribution If you write test questions for an Associate or Specialist exam, you should be familiar with the training material so that you have an idea of how SAP envisions the respective role.

If you write test questions for a Professional exam you should have worked for at least four years in the respective SAP solution domain.


Target audiences for SAP exams We want to test whether candidates know enough in the domain of the certification exam. Knowing enough is, of course, hard to define but normally you know it when you see (or dont see) it. Someone knows enough in the exam domain if she or he can contribute positively to a project. Someone knows enough if she or he does not cause damage due to lack of knowledge in the exam domain.

We are not trying to identify the top performers. If we can identify the ones who know enough, the ones who are sufficiently qualified, we have achieved our goal.

This means that you should write questions that you would ask someone to find out whether she or he knows enough. Your questions work like a filter. If the questions you ask are too easy, they will not act as a filter because everyone will answer them correctly. If your questions are too difficult, only a few will pass your filter. Therefore, write the questions that you think perhaps half of the candidates would answer correctly.


Relevance You should write test questions that matter. They should require the exam candidate to think. They should not just require the candidate to retrieve trivia from her or his memory.

If you want to write a question about something, think first about what the lack of knowledge could do to an implementation project:

Would the project be delayed or would customer satisfaction be reduced? If so, the chances are that you

are on to something that is relevant. Does the subject of your question come up in almost every project? If so, the chances are that you

are on to something that is frequent. Anything that is somewhere else might be important but rare, so it cannot be considered typical.


A few words of caution Be aware of an availability bias:

Just because you experienced something in your last project does not mean it is important in most other

projects. Just because you saved the day with a particular insight does not make this a good subject for a test

question. Kudos to you but it may be too exotic.

Be aware of opinion: exams force a candidate to make clear choices by selecting one, two, or three correct answers. There is no place for discussion. If something is a matter of opinion, the chances are that the correct answer would start with It depends We cannot fail someone just because she or he has a different opinion. Would you like to fail an exam because you have a different opinion?


Conciseness Certification exams do not instruct. They test whether someone knows enough or is competent enough. If she or he did not learn something she or he was supposed to have learned, it is too late. Therefore, do not explain general mechanisms, concepts, terms, etc. If you want to use these terms in a test question, then you expect that the exam candidate already knows this.

You have taken certification exams. Every minute mattered. Dont waste your time writing lengthy questions. Dont waste an examinees time by forcing her or him to read a bunch of words that convey little or no information.

More words mean more opportunities for confusion. If you use as few words as possible, you can reduce the chances of causing confusion. Write your questions to the point. We dont window dress questions.

Language Your submissions must be in English. Preferably, the questions should also be grammatically correct.


Question types SAP certification exams currently use selected response questions: exam candidates select one, two, or three answers from a set of answers. The question format of selected responses helps SAP ensure fairness for native and non-native speakers of English alike.

We do not use questions that have only two answers. This means that you cannot ask questions like Can you transfer data models? or Are data models transferable?.

Multiple choice (MC) The question is presented with four answers of which one is correct. The candidate chooses the correct answer by selecting a radio button that is displayed in front of an answer.

Multiple response (MR) This question type has either four or five answers.

Dichotomous scoring Each question yields one point if the correct answers and only the correct answers have been selected. There is no partial credit for selecting a subset of correct answers.


Parts of a question A question consists of two parts:

Stem Answers

At a minimum the stem consists of one sentence: the question. The stem can also be a composite: a scenario or mini story and the final question.

A scenario describes a specific business situation, for example events, data, actions taken, actions planned, and errors. It does NOT explain concepts or terms or anything the candidate is supposed to know at the time of the exam.


Multiple choice In SAP exams multiple choice (MC) questions always contain four answers. Of these one, and only one, is correct. The candidate can only select one answer.

When you submit a MC question to the SAP Crowdsourced Certification Questions Initiative the first answer must always be the correct one. This is simply to avoid confusion. In an SAP certification exam, answers are always shuffled.

The shuffling of answers has one major consequence: you cannot use anything like All of the above, None of the below. Since answers are shuffled, an answer will be presented at any given position.


Multiple response In SAP exams, this question type comes in two flavors:

MR1: two correct answers, two incorrect answers, a total of four answers. MR2: three correct answers, two incorrect answers, a total of five answers.

We always tell candidates how many answers they have to select. As already discussed under dichotomous scoring, candidates receive one point if they select the correct answers. We do not award partial points.

Multiple response (MR1): four answers, two correct Choose this question type if there are two parts to a correct answer, for example two ways of accomplishing something, two settings that must be made, two conclusions that can be drawn.

Multiple response (MR2): five answers, three correct Choose this question type if there are three parts to a correct answer.


Item writing process We recommend that you adhere to the following process.

Plan

Currently you have to choose the item type first. If you want to change your mind later, simply go back to the web browser.

After you select the item type, you can select a topic from a list, for example


1. Pick one topic you are very familiar with, for example, Administering Data Models. 2. Think about a relevant task that you frequently perform in this area. 3. Write this task down, for example, Review the history of activated models. 4. Determine the knowledge or skill required to perform this task successfully. 5. Pick a result of your analysis and write a question about it.

Write Start with the stem. Get a very clear idea about what you want to know from the candidate. The clearer the stem, the clearer the answers will be. If you are fuzzy about what you want to know, chances are that your answers will be fuzzy too. We try to avoid fuzzy questions!

After you have written the stem:

1. Write the correct answer or the correct answers, depending on the question type that you chose. 2. Enter the source information for the correct answer(s). 3. Write the incorrect answers. 4. Verify everything.

Submit You are now good to go. After you submit the question, you will have a chance to review everything. Once you are satisfied, submit the question. You can see your submissions but you cannot change them.


Item writing rules

We ask you to follow eight important rules. If you follow these rules all of them the odds are pretty good that your submission will be accepted.

1. Specificity 2. Content and level 3. Fairness 4. Brevity 5. Proper English 6. No negatives 7. Parallel answers 8. Terminology

Before we continue and describe the rules in detail, let us outline some general best practices that will increase your chances of submitting a question that we will accept.


Overview

Stem No matter how many sentences you write for the stem (and it should not be too many), the last one must always be the question. While this may sound obvious, we receive quite a few submissions that do not contain a question. The least we expect is a complete sentence that ends in a question mark. The word order of a question sentence is of course a bit different from a statement. A sentence that ends in a colon or with continuation points () is simply not a valid stem. This will almost always lead to your submission being rejected.

Answers Every single answer must be a grammatically adequate answer to the question. Every single answer must be self-contained and not refer to another answer in any way.

Correct answers A correct answer is demonstrably and clearly correct. An answer must be correct beyond reasonable doubt. It is not sufficient for an answer to be almost correct, for example, because all other answers are false.

Incorrect answers Incorrect answers are clearly false in a non-trivial sense. In selected response questions, the incorrect answers serve as distractors for a candidate who is not competent (enough). Even incorrect answers have to be fair. Therefore, we do not allow answers as distractors that are too true to be true. Incorrect answers can be hard, but they must not be sneaky or malicious. From a practical point of view, you must not invent, for example, system options, method names, transaction names, and so forth.


Source information We use this information to easily verify the correctness of the answer(s). Be as specific as possible when you provide this information. Only use sources that are, or can be, available to the general public.


Rules Rule 1: Specificity The question is non-trivial and specific to the exam and usually starts with one of the six question pronouns described below. The question stem avoids anything like, Which of the following statements are true? First, a question stem like this could be used in any exam. There is nothing specific about it. Second, the question stem is sometimes called a stimulus. A stimulus is supposed to trigger something. In the case of certification questions, the stimulus is supposed to evoke a specific response, for example, a definition or a rule, or a procedure to accomplish a task or to figure out how to solve a problem. A stimulus should make the examinee think about a particular topic. Therefore, a question stem like Which of the following statements is true? does not evoke anything. The examinee must read the answers first to get an idea of what we wanted to ask. In our experience, the answers in this case are a hodgepodge of things that lack clarity. We think that this is unfair to examinees. We are asking examinees to take a high-stakes exam. The least we can do is to confront the examinees with well-thought-out questions; difficult maybe, but to the point and fair.

Never try to submit questions like Which of the following statements is true? or Which of the following statements are not correct? or Which of the following statements regarding authorizations is false? We cannot, and we do not, want to list all possible ways of phrasing this kind of poor excuse for a question. You submit it, we will reject it!

Rule 2: Content and Level The question fits the topic, is representative of the subject matter, and is adequate for a candidate of average competence for a particular exam.

Dont write trivial questions. Dont write questions only a handful experts on the planet could perhaps answer.


Rule 3: Fairness The question is fair, clear, suitable for an international audience, and gender-neutral. The question avoids jokes, idioms, and colloquialisms.

Fair: Dont write honey traps. Clear: Dont cloud what you want to say by using many adjectives, etc. Write plain and simple

questions! International audience: SAP exams are taken in over 50 countries. You cannot take anything for

granted. Gender-neutral: No he, no she, no John, no Jane. No idioms: Not even once in a blue moon can you write a question that asks for an eleventh hour

change Colloquialism: Be precise in the terminology you use.

Rule 4: Brevity The question is written concisely with as few words as possible. Most words are put in the stem and not in the answers. Do not explain anything you expect the average competent candidate to know.

Rule 5: Proper English Write in Standard English, using the active voice, and present tense. The stem always ends in a question mark (?).

Rule 6: No Negatives Negatives can be confusing or are easily overlooked. Avoid negatives such as not in any combination, for example, does not, cannot, could not, should not, or must not. Examples: Positive and conforming to the rules: Which method can you use to create an index? Negative and violating the rules: Which method can you NOT use to create an index? Negative and violating the rules: Which method should you avoid when you create an index?

There is one exception: When you are writing a troubleshooting question, you could describe a scenario or an action that fails and ask about the reason(s), you could describe a failed action and ask how to correct this, or you could describe a failed operation and ask how the failure could have been prevented. In these cases you might see the need to use one negative. If you use a negative in the stem you cannot use negatives in answers.


Rule 7: Parallel answers Answers should not give away a clue to the question. Therefore ensure that:

All answers match the question. The wording of the answers does not hint at their correctness or incorrectness. All answers are of roughly equal length. All answers are phrased in a parallel way. All answers are meaningful and do not refer to other answers. All answers are plausible and nothing is made up. Answers such as All or None of the above/below are never used.

Try keeping the answers as short as possible. If all the answers have words in common try to put these into the stem.

Rule 8: Terminology The terminology (SAP and otherwise) must be correct. Do not make up anything, for example, do not invent method names, transaction names, system settings, and so forth. The capitalization of terms must follow the official guidelines. When you refer to transactions, use the descriptive name of the transaction and put the transaction code in parentheses, for example, the Workbench Organizer (SE09) transaction. You can find approved names of all SAP products at SAP Branding Tools Naming. You can use abbreviations after you introduce the long form first. Put the abbreviation in parentheses after the long form. Then you can continue to use the abbreviation. Typically, you want to do this in the question stem. Since answers are presented in random order, it is usually neither necessary nor useful to introduce abbreviations in answers.


Examples The following examples provide you with some ideas on how to write questions.

Pronouns Most of your questions will start with one the following question pronouns:

1. What 2. When 3. Where 4. Which 5. Why 6. How

What questions What can you use to update data source settings on the system database? What information must you collect before you can run an XYZ report? What data is included in the sales report?


When questions When would you install the Advanced Adapter Engine decentrally? When are xyz tables created?

Where questions Where is this change reflected? Where do you set the xyz parameter?

Which questions Which of the following statuses should a work item have if you want to archive it? Which of the following must exist before an agent can create an automatic billing document? Which indicators must you define to enable a performance trace?

You might say But didnt they forbid the use of Which of the following? We do not allow things like Which of the following statements is true? You can use Which of the following if you need to limit the choices you list in the answers. For example, if you want to ask for system options but cannot list them all, use Which of the following system options can you activate to That way a candidate knows that the list provided is not exhaustive.

Why questions Why would you use the test tool of the Business Object Repository (SWO1)? Why would you build an SAP BusinessObjects BI universe on top of SAP HANA using the Information Design Tool (IDT)?

How questions How do you implement? How does the system behave if you have an Idle Connection Timeout setting? How can you transport content?

Keep in mind: All answers must be grammatically correct answers to the question pronoun you choose.


Single or multiple sentence stems Single sentence stems This is the shortest way of asking a question. You start with one of the pronouns listed in the previous section and complete the sentence following the rules.

What is the valid sequence for processing scheduled meter readings? When would you consider setting up an SAP data archiving project?

You would probably use single-sentence questions if you are interested in finding out whether a candidate has understood a concept, a procedure, or a process. Typically, short questions test memorization of things because it is difficult to describe a situation and ask a question about this in one (comprehensible) sentence. Although it is always possible to write one long and winding sentence, this would go against the rules described above. Long and winding sentences are more difficult to understand for linguistic reasons alone. Write in terse prose. Split up what you want to say into logical units rather than trying to fit them into one monstrous construction. Keep in mind that a candidate has about two minutes to read, comprehend, and answer an exam question.

Sometimes you may want to stress the context of a question. In this case you could put the context first. For example In IT Service Management, how can you.

Multiple sentence stems You use multiple sentences to describe a specific situation before you ask the question. First, set up a scenarioa typical and specific business situation that includes events, data, actions taken, actions planned, errors, and so on. Second, ask the question (see single sentence question above). Common situations would be:

Describing a specific (business) situation, goal, or error Describing an action or actions Describing (business) requirements

Examples 1. Description of a specific [business] situation, goal, or error 2. One 6W question


Billing fails to produce a billing document due to a missing move-in meter reading. What can you configure to notify the back office of this failure?

A tariff requires that for the same contract account, the consumption from the water meter is used to bill the water consumption charge and the waste water consumption charge. How can the water meter be installed to meet this tariff requirement?

1. Description of an action or actions 2. One 6W question about consequences or actions

You change the title of a Crystal Reports object using the Central Management Console. Where is this change reflected?

Your customer has created a custom program, Z_ CUST_ PRG, that updates custom table Z_ CUST_ TBL. The Firefighter ID is used to execute the Z_ CUST_ PRG program during a Firefighter session. How do you ensure that the Firefighter log includes changes made to the Z_ CUST_ TBL table?

1. Description of an action or actions 2. Description of unexpected consequences 3. One 6W question about problem resolution, analysis, or causes

A user has started an instance of a workflow. The first step of the workflow is an approval step that should have created a work item for the manager. However, the manager did not receive the work item. What should you do first to analyze the cause of this issue?

You create a workflow that starts when the event EQUI. Created is raised (a new equipment is created). The first step is to display this equipment using the EQUI.Display method. However, this method is not receiving the equipment object. What binding would you add to resolve this problem?


Answers Answers are always presented in random order, that is, you cannot rely on a specific answer to be on top or at the bottom. Hence, pseudo answers such as All of the above, All of the below, None of the above, and so forth are meaningless and therefore not allowed. An answer such as All of the above would normally violate the rule that all answers must be linguistically correct answers to the question asked.

According to rule 6 regarding parallel answers, all answers must match the question from a content or language point of view. All answers should be of roughly equal length, phrased in a parallel way, be meaningful, and stand for themselves.


Terminology

Term Meaning Exam question

An exam question comprises the elements: Question stem Correct answer(s) Incorrect answers Source information for correct answers.

Correct answer The answer(s) a candidate must select to score a point.

Dichotomous scoring A candidate scores a point if the candidate selects the required number of correct answers. For multiple choice questions, this is 1 answer. For multiple response questions, this is either two or three answers. Failure to select the correct number of answers results in zero points. No partial credits are awarded.

Incorrect answer A false answer, also called a distractor

Item Short for test item, the same as an exam question

Multiple choice question (MC)

A question with: One correct answer Three incorrect answers

All answers are displayed with one radio box; thus it is only possible to select one answer.

Multiple response question type 1 (MR1)

A question with: Two correct answers Two incorrect answers

The candidate must select the two correct answers to score one point.

Multiple response question type 2 (MR2)

A question with: Three correct answers Two incorrect answers

The candidate must select the three correct answers to score one point.

Question stem The question, which consists of: Zero or more sentences that describe a scenario Exactly one grammatical question

The minimum length is 50 characters.

Validation or source information

Precise and accurate information that must be supplied to enable SAP to validate the correctness of an answer.


Exercise No.

Example

Rule 1: Specificity

Rule 2: Content and level

Rule 3: Fairness

Rule 4: Brevity

Rule 5: Proper English

Rule 6: No negatives

Rule 7: Parallel answers

Rule 8: Terminology

1 Afaria installation pack doesnt consists of

2 Choose main functionalities of Data Orchestration Engine.

3 What will you use to fulfill this requirement?

4 What can be used to filter a table on multiple selection criteria?

5 Which of the following statements regarding CUA is correct?


No.

Example

Rule 1: Specificity


Rule 3: Fairness

Rule 4: Brevity




Rule 8: Terminology

6 Connections allow Unwired Server to communicate with data sources. At what level in the environment are connections managed and maintained?

7 Certain elements of a system landscape are crucial for the availability of the system. These elements are considered Single Points of Failure since if they stop working, the entire system will be unavailable. What elements are considered Single Points of Failure?

8 Which of the following statements are not true?


No.

Example

Rule 1: Specificity


Rule 3: Fairness

Rule 4: Brevity




Rule 8: Terminology

9 Question: How can you identify the pricing function within a trace file? Answers: You cannot identify the function in a trace file.

10 Question: What happens when the config file is accidentally deleted? Answers: All nodes in the multiplex will shut down. All writers in the multiplex will shut down. Another node must be manually promoted to


No.

Example

Rule 1: Specificity


Rule 3: Fairness

Rule 4: Brevity




Rule 8: Terminology

11 Question: Can additional indexes be useful for improving performance? Answers: Yes, because a database will use all information. No, because one index is sufficient. Yes, because indexes can be combined by the optimizer. No, because the optimizer cannot combine multiple indexes.

Documents

Question Guidelines for SAP Certification