ROSAI: Role-Oriented Skills Assessment for AI — A Framework for Role-Relative AI Readiness
Author: Prasanjit Saha
Version: 1.0
Repository: Zenodo
DOI: 10.5281/zenodo.22801326
ROSAI Framework · Version 1.0
ROSAI: Role-Oriented Skills Assessment for AI
A framework for measuring AI readiness relative to what a role actually expects.
ROSAI — Role-Oriented Skills Assessment for AI — is a framework developed by Prasanjit Saha for assessing professional AI readiness relative to the mix of responsibilities expected from a role.
AI capability is multidimensional. AI readiness is role-relative. ROSAI represents a target role as a blend of responsibilities, dynamically adjusts the importance of seven AI capability dimensions, and reports capability separately from Evidence Strength and Assessment Confidence.
Developed by Prasanjit Saha · Version 1.0 · September 2026
Research records: Zenodo · SSRN
Framework at a glance
01 · Role composition
Six responsibility families
Product / Business · Software Engineering · Data / ML · Operations / Transformation · Design / UX · Leadership / Governance
02 · Capability model
Seven AI capabilities
AI Fluency · Applied AI Judgement · Build & Execution · Evaluation & Reliability · Data & ML Capability · Responsible AI & Governance · Business & Organizational Impact
03 · Dynamic weighting
Importance follows the role
The role composition changes the relative importance of the seven capabilities.
04 · Interpretation
Three complementary signals
ROSAI Score + Evidence Strength + Assessment Confidence. Each answers a different question.
AI capability belongs to the individual. AI readiness exists in relation to a role.
How well does this person's AI capability fit what this role actually expects?
The problem with “AI-ready”
“AI-ready” is increasingly used as a professional label, but it has no single practical meaning.
For one person, AI readiness may mean using generative AI effectively for research and productivity. For another, it may mean building RAG systems, integrating AI APIs or deploying ML workflows. A Product Manager may be expected to identify strong AI use cases and evaluate product quality. A senior leader may need to make sound decisions on AI strategy, governance, risk and adoption.
These can all be legitimate forms of AI capability. They are simply not the same capability.
Job titles are imperfect proxies for responsibility. Two people called “Product Manager” may be expected to perform very different combinations of product, engineering, data, transformation and leadership work. ROSAI therefore evaluates readiness against the responsibility mix of the target role.
The same person can be highly AI-ready for one role and less AI-ready for another without their underlying capability changing.
ROSAI does not assume AI-skills frameworks did not already exist
AI literacy, AI competency and professional AI proficiency already have substantial research and framework literature. Existing work includes validated AI-literacy scales, workforce competency frameworks, role-sensitive professional skills models and role-adaptive assessments. The references below identify the work discussed in the formal paper.
ROSAI does not claim to have invented multidimensional AI capability or role-aware assessment.
The specific contribution of ROSAI
ROSAI proposes an assessment architecture built around four combined design choices:
Continuous role composition
A role is represented as a percentage mix of responsibilities rather than one fixed occupational label.
Dynamic role-derived capability weighting
The role composition mathematically determines the relative importance of the capability dimensions.
Capability, evidence and confidence are separate
Demonstrated capability, supporting evidence and confidence in interpretation are not collapsed into one number.
Role-critical safeguards
A severe weakness in a capability identified as core to the role cannot always be averaged away by unrelated strengths.
This is a proposed contribution, not a claim that ROSAI is the first framework in history to combine these ideas. Version 1.0 makes the methodology explicit enough to be tested, challenged and improved.
A role is a mix of responsibilities, not just a job title
ROSAI models the target role as a six-component vector whose values sum to 100%.
| Responsibility | Example share |
|---|---|
| Product / Business | 50% |
| Software Engineering | 20% |
| Operations / Transformation | 15% |
| Leadership / Governance | 10% |
| Data / ML | 5% |
| Design / UX | 0% |
This could represent a hands-on AI Product Manager. It is not intended to be the standard profile for every Product Manager.
Same capability profile
Product-heavy
Different role-relative reading
Data/ML-heavy
Different role-relative reading
Leadership-heavy
Different role-relative reading
What ROSAI measures
CAPABILITY 01
AI Fluency
Practical understanding of AI concepts, capabilities, limitations and common solution patterns sufficiently to work effectively with AI.
CAPABILITY 02
Applied AI Judgement
Ability to decide when, where and how AI should be used, including trade-offs across value, accuracy, latency, cost, privacy, risk and human oversight.
CAPABILITY 03
Build & Execution
Ability to move from an AI idea toward a working prototype, workflow, integration, production implementation or managed delivery.
CAPABILITY 04
Evaluation & Reliability
Ability to determine whether an AI system is reliable enough for its intended use through evaluation, validation, monitoring, edge-case testing and controls.
CAPABILITY 05
Data & ML Capability
Ability to understand and work with data and machine-learning concepts at the depth required by the role.
CAPABILITY 06
Responsible AI & Governance
Ability to recognize and manage privacy, security, permissions, fairness, explainability, auditability, human oversight, regulatory exposure and responsible deployment.
CAPABILITY 07
Business & Organizational Impact
Ability to connect AI activity to adoption, productivity, customer outcomes, cost, revenue, workflow redesign, prioritization and organizational change.
ROSAI deliberately separates building something with AI from creating value with AI.
How role-relative weighting works
The target role composition is combined with a versioned role–capability model to derive the relative importance of each capability.
W = R × M
- R = target role-composition vector
- M = versioned role–capability model
- W = role-adjusted capability-weight vector
Base ROSAI Score = Σ (Capability Weight × Capability Score)
A role-heavy requirement for Build & Execution will assign more weight to that dimension. A leadership/governance-heavy role may assign more importance to Responsible AI & Governance and Business & Organizational Impact.
Why a weighted average is not enough
Weighted averages can hide severe gaps. A candidate can score extremely well in several dimensions and mathematically compensate for a major weakness elsewhere.
ROSAI therefore includes a non-compensatory safeguard for capabilities that the target role identifies as core. In Version 1.0, the public methodology exposes the principle, while operational thresholds remain versioned implementation parameters.
A severe gap in a capability that the role itself defines as core should not always be averaged away by unrelated strengths.
One score is not enough
ROSAI Score
How well does the assessed capability profile fit this role?
Represents role-adjusted AI capability.
Evidence Strength
How strongly are the relevant capability claims supported?
Can include live products, repositories, demos, design documentation, case studies, measurable outcomes, publications, employer work or references.
Assessment Confidence
How confidently can the available assessment signal be interpreted?
Considers completion, consistency, scenario alignment, response quality and supporting evidence.
ROSAI Score + Evidence Strength + Assessment Confidence
Role-oriented interpretation profile
Role-oriented interpretation profile
Evidence Strength and Assessment Confidence do not directly increase or decrease the ROSAI Score in Version 1.0.
How ROSAI is assessed in SCORE-AI
SCORE-AI is the first reference implementation. It uses an approximately 15-minute, 18-question assessment combining multiple signal types rather than relying entirely on self-rated proficiency.
- Define role context
Role mix and responsibility level. - Answer structured capability questions
Decisions, usage, evaluation, production readiness, practical experience and Data/ML maturity. - Provide optional evidence
Relevant proof of work. - Complete adaptive scenarios
Three short practical cases selected according to the dominant role mix.
Structured questions provide consistency and scalability. Adaptive scenarios provide an applied reasoning signal.
Applied judgement, not AI vocabulary
The adaptive cases are evaluated against four common criteria:
- Context Framing
- Decision Quality
- Evaluation / Control Discipline
- Role-Specific Execution Depth
The grading model is intended to act as a rubric executor rather than an unconstrained judge. It is instructed to score explicit content, avoid rewarding jargon and avoid inferring experience that the respondent did not state.
The operational assessment stores framework, rubric, prompt and grader-model versions to support later reproducibility testing.
ROSAI v1.0 — high-level algorithm
Input
Target role composition · Responsibility level · Structured responses · Practical-experience signals · Optional evidence · Adaptive scenario responses
- Derive seven capability scores
- Generate role-adjusted capability weights from the role composition
- Calculate weighted capability fit
- Apply role-critical safeguard
- Calculate Evidence Strength separately
- Calculate Assessment Confidence separately
Output
ROSAI Score · Evidence Strength · Assessment Confidence · Seven capability scores · Role-relevant strengths · Development priorities
What ROSAI publishes — and what it does not
Public methodology
- Construct definitions and responsibility families
- Seven capability dimensions
- Continuous role-composition model
- Dynamic-weighting and capability aggregation equations
- Role-critical safeguard principle
- Evidence Strength and Assessment Confidence dimensions
- Adaptive-case rubric categories
- Validation roadmap and versioning principles
Operational parameters
Not publicly disclosed in Version 1.0:
- Exact role–capability coefficients
- Option-level scoring vectors and item-level answer values
- Exact guardrail thresholds
- Exact consistency penalties
- Evidence coefficients
- Anti-gaming heuristics
ROSAI distinguishes framework transparency from answer-key disclosure. The public paper exposes the methodology needed to understand, critique and cite the framework, while keeping the operational answer key from becoming a gaming guide before validation is complete.
Development should also be role-relative
A person's lowest absolute capability is not automatically the most useful thing to improve.
Development Priority ∝ Role Importance × Capability Gap
A Data/ML gap may be highly consequential for an ML-heavy role and far less consequential for a leadership role where Data/ML depth has low importance.
Current status and limitations
ROSAI Version 1.0 is a practitioner-designed framework and testable methodology. It has not yet undergone psychometric validation or peer review.
It is currently not:
- A professional certification
- A measure of intelligence
- A guarantee of job performance
- A replacement for interviews or technical assessment
- A psychometrically validated employment-selection test
- An industry standard
- A universally accepted definition of AI readiness
The framework contains formal equations. Formalization makes the methodology explicit; it does not make the methodology empirically validated.
How ROSAI should evolve
ROSAI should evolve through evidence rather than author judgement alone.
- Expert content review
- Pilot data collection
- Reliability and grader-agreement testing
- Construct and known-groups validation
- Criterion-related validation
- Fairness and bias analysis
- Recalibration and benchmarking
A framework for AI readiness should itself be evidence-driven.
What future validation should test
- Role sensitivity
Changing the role composition should alter readiness interpretation in a direction consistent with that role’s capability expectations. - Construct separation
Evidence availability should change confidence in interpreting a capability claim without automatically becoming capability. - Non-compensation
A severe deficit in a role-critical capability should not be fully hidden by unrelated strengths. - Applied alignment
Strong structured responses should generally align with reasoning demonstrated in adaptive scenarios. - Practical usefulness
Role-weighted development priorities should be more actionable than recommendations based only on the individual’s lowest absolute capability.
Try the framework
SCORE-AI — Skills, Capability, Outcomes, Role-fit & Evidence for AI — is the first public implementation of ROSAI.
Define a target role mix, complete the assessment, provide optional evidence and receive a role-oriented capability profile: ROSAI Score, Evidence Strength, Assessment Confidence, seven capability scores, strengths, development priorities and a downloadable report.
Free · ~15 minutes · No sign-up · No sign-in
Research & Publication
ROSAI Version 1.0 is publicly documented as a practitioner-designed framework and testable methodology. The same framework paper is available through both Zenodo and SSRN for citation, discovery and long-term access.
ROSAI: Role-Oriented Skills Assessment for AI — A Framework for Role-Relative AI Readiness
Author: Prasanjit Saha
Platform: SSRN
SSRN Abstract ID: 7479102
DOI: 10.2139/ssrn.7479102
These are two public records of the same ROSAI v1.0 framework paper. Their presence on Zenodo and SSRN should not be interpreted as peer review or empirical validation.
Read and cite the ROSAI Framework
Preferred / original repository citation
Saha, P. (2026). ROSAI: Role-Oriented Skills Assessment for AI — A Framework for Role-Relative AI Readiness (Version 1.0). Zenodo. https://doi.org/10.5281/zenodo.22801326
All-versions Zenodo DOI: 10.5281/zenodo.22800584
SSRN citation
Saha, P. (2026). ROSAI: Role-Oriented Skills Assessment for AI — A Framework for Role-Relative AI Readiness. SSRN. https://doi.org/10.2139/ssrn.7479102
Selected references
Bibliographic entries from the formal ROSAI paper. The paper contains the definitive bibliography.
- Long, D., & Magerko, B. (2020). What is AI Literacy? Competencies and Design Considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–16. Read source ↗
- Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, 100041. Read source ↗
- Laupichler, M. C., Aster, A., Haverkamp, N., & Raupach, T. (2023). Development of the “Scale for the assessment of non-experts’ AI literacy” — An exploratory factor analysis. Computers in Human Behavior Reports, 12, 100338. Read source ↗
- Carolus, A., Koch, M. J., Straka, S., Latoschik, M. E., & Wienrich, C. (2023). MAILS — Meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies. Computers in Human Behavior: Artificial Humans, 1(2), 100014. Read source ↗
- Lintner, T. (2024). A systematic review of AI literacy scales. npj Science of Learning, 9, 50. Read source ↗
- Markus, A., Carolus, A., & Wienrich, C. (2025). Objective measurement of AI literacy: Development and validation of the AI competency objective scale (AICOS). Computers and Education: Artificial Intelligence, 9, 100485. Read source ↗
- The Alan Turing Institute, Innovate UK, Department for Science, Innovation and Technology, Digital Catapult, Science and Technology Facilities Council, British Standards Institution, & Alliance for Data Science Professionals. (2026). AI Skills for Business Competency Framework (Version 3.0.0). Zenodo. Read source ↗
- Skills England. (2025–2026). AI skills for the UK workforce. UK Government. Read source ↗
- appliedAI Institute for Europe. (2026). AI Skills Framework — Defining, Developing & Assessing Professional AI Competence. Read source ↗
- Miao, F., Shiohira, K., & Lao, N. (2024). AI competency framework for students. UNESCO. Read source ↗
- Miao, F., & Cukurova, M. (2024). AI competency framework for teachers. UNESCO. Read source ↗
- OECD & European Commission. (2026). Empowering Learners for the Age of AI: An AI Literacy Framework for Primary and Secondary Education. OECD Publishing. Read source ↗
- Nguyen Trieu, H. (2026). CFTE AI Proficiency Framework. Centre for Finance, Technology and Entrepreneurship. Read source ↗