Model Cards
Model Cards
Model cards refer to structured documents that describe how an AI model works, what it was trained on, how it performs, and where it should or shouldn’t be used. They typically list out intended use cases, performance metrics, evaluation results, results of bias testing, limitations and failure modes, and ethical considerations. Model Cards were originally formalized by researchers at Google and have since become a widely adopted practice for documenting specific across the AI ecosystem and lifecycle.
Compliance
Model Cards help comply with NIST and ISO frameworks, which encourage documentation and transparency. They also align with the EU AI Act’s requirements for documentation and transparency.
In practice
Model Cards are primarily used by AI companies in the form of system cards and model documentation for major releases, for vision and ML systems, and as transparency notes and documentation of responsible AI. They are also included in the form of README sections or as separate documentation files.
However, given that there is no single regulatory approach to Model Cards, coverage is often selective and the depth of information provided in Model Cards tends to vary widely from organisation to organisation. Some offer detailed reports, some offer high-level summaries, and some fall in between.
High-effort, detailed reports typically comprise detailed evaluations across multiple benchmarks, explicit bias and safety analyses results, and clear mentions of limitations. Low-lift, minimal compliance-styled summaries offer up generic statements about risks, limited or no quantitative evaluation, vague usage guidance, and no real clarity on the bias and limitations.
Embedding Responsibility and Ethical Practices
Model Cards solve a major problem. People who use AI tools don’t always have a full understanding of the tool, or what it can and cannot do. Model Cards help bridge this gap by offering transparency, demonstrating accountability, and equipping users with usability guidance that can help them make informed decisions on whether or not to trust outputs.
