Tips for Setting Question Difficulty

QA’s Content Engine allows you to build exams that greatly affect your team’s skill score. It is important that you give the exam questions you create the correct difficulty rating so that our skill score engine can truly reflect the skill level acquired by your team. The guidelines below help you set the right difficulty for custom questions you create in Content Engine.

Consider the following guidelines when you are setting the question's difficulty. These guidelines do not consider the particular tools or task that the question refers to, but just offer general suggestions.

Beginner questions likely focus on concepts and a high-level understanding of a topic. Beginner questions may:

• focus on a tool's or service's purpose and use cases
• test general concepts related to the service (authorization for a security service or high availability for a configuration management service)
• test on price structure, basic service compatibility, and features or components

Intermediate questions transition from conceptual knowledge to the basic application of "hands-on" knowledge. Intermediate questions on a service may:

• focus on configuration choices to achieve the desired outcome
• test on how to solve problems using a service
• require an understanding of best practices
• require experience in the related domain of the service, such as networking, coding, encryption, or data analysis

Advanced questions tend to be far more detailed and complex than intermediate questions. Advanced questions may:

• focus on optimizing the efficiency of multiple integrated services
• ask for the best choice rather than the correct choice
• require considering multiple factors, such as cost, performance, and time, to find the best answer

Other Tips:

• Consider the area of knowledge the question tests when you set the difficulty level. Conceptual questions on specialized areas (such as machine learning or DevOps automation) are likely more difficult.
• Consider how long it might take a person to gain this knowledge in the field. If someone with six months of experience could answer the question, it is likely a beginner question. If someone with 5 years of experience would need to review the question closely, it's probably advanced.
• Remember these are just guidelines, and there is no one-size-fits-all method for setting the difficulty rating.