Students Learn How Open Source Tools Turn AI Demos Into Reliable Systems

From laptop demo to production

The event, titled Open Source AI: Building Reliable AI Systems, was hosted and organized by Susan Malaika (IBM, Responsible AI at LF AI & Data), in collaboration with SJSU's Career Center and Responsible Computing Club. Jennifer Hernandez, IBM's Client, University & Community Engagement Leader, helped organize it. The recap was written by Julia Husainzada, a data science student and president of the Responsible Computing Club.

The starting point was simple: building an AI demo has never been easier, but turning it into a scalable system is the hard part. Each of the four tools covered a different piece of that problem: making LLM calls reliable, keeping AI safe, building it securely and grounding it in reliable data.

Mellea and ASAGO: reliable calls and testable safety

Basel Shbita presented Mellea, a Python library that sits between the application and the model and treats the LLM as a building block. Tasks are split into smaller steps, each with its own instructions, requirements and sampling strategy. Every call follows an instruct, validate, repair pattern and must return the data type the developer defined. The Mellea team calls this generative computing. According to the recap, it lets small open-weight models do big-model work at a fraction of the cost, makes models and inference engines easy to swap, and puts behavior under the control of the developer's program rather than a fragile prompt. It can be installed with pip install mellea.

Muneeza Azmat, a Senior ML Engineer on Red Hat's AI safety team, presented ASAGO, an open source project that automates the pipeline from policy to production. It treats the policy document as the ground truth, maps its sections to established AI risk taxonomies, generates risk scenarios based on the tools an agent can access, and translates them into tests for red-teaming tools such as Garak. Because each test traces back to the policy, compliance teams get traceability and a measure of how much of the policy is covered.

Bob and Docling: secure development and dependable data

James Busche showed Bob, IBM's AI development environment, as an example of developers shifting from writing every line of code to conducting the orchestra. Bob acts only with the permissions the developer grants. It supports security validation from the start, global or project rules such as requiring test cases for new code, onboarding to unfamiliar codebases and documentation generation. It has three modes, Ask, Plan and Agent, and starts with small, inexpensive models, stepping up only when needed.

Ming Zhao presented Docling, an open source project started by IBM Research and now hosted by the LF AI & Data Foundation. It converts formats such as PDFs, Excel files, HTML, images and audio into AI-ready formats such as JSON, preserving layout and tables. Its chunks keep metadata and bounding boxes, so a RAG system can point to where in a document an answer came from. The recap cites a small vision model of about 258M parameters that processes a page in about 4 seconds, and says Docling can be around 50x more cost-effective than running every page through a large vision model.

What students were told to do next

The speakers' advice was practical. Shbita urged students to break problems into subproblems. Azmat said ASAGO holds biweekly meetings and has Slack channels where anyone can contribute. Busche suggested experimenting for free, since open source LLMs can run on a personal laptop.

Zhao said validation systems are likely to become one of the largest sectors in AI over the next few years, because many organizations adopted AI without properly validating its work. Malaika advised students to join a group or open source project, take an internship or build something real, noting that companies are looking for practical experience.

For students and early-career developers, the message of the event is that reliability skills, such as structured prompting, policy-based testing, secure development and data validation, can be learned through open source projects that welcome contributors.