Production Open Source Chat with Work Instructions

Published: 2025-03-06
Updated: 2025-03-09
Author: Chuck Boecking
Discussion: https://team.chuck-stack.org/t/production-open-source-chat-with-work-instructions/81
Status: Drafted (more coming...)

Create Your First Work Instruction Book with Chat

The purpose of this post is to help you get your first work instructions published with chat capabilities.

Here is the benefit story:

You will thrust your AI capabilities ahead of 90% of all organizations in a single step.
You will create a platform for building future automation around your work instructions.

It is important to note that I am a champion of small to medium organizations. You are my target audience. This post was written for you!

As a result of this article, you can:

deploy a production quality system in minutes
use a framework to publish work instructions with zero additional effort
connect to most (if not all) of the world's most powerful language models (LLM) with no code change
improve your work instructions with AI assistance at near real-time speed where changes are based on actual human usage and feedback
set your organization on a path to achieve the world highest quality awards

Problem Statements

Here are the problem statements we want to address:

Problem #1: AI is complicated to understand much less create a solution around AI.

Problem #2: The vast majority AI tutorials are based around writing code and they are limited to just a few LLMs and a few scenarios.

Problem #3: Deploying code to production is complicated if you do not have the expertise.

Problem #4: Creating a plan for AI success in a small and medium organization is not obvious.

Getting Started

Here is what we will do:

We are going to implement the system you see here. Please watch the videos on this link first to see the big picture if you have not done so yet.
We will configure a Linux server to deploy existing open source applications (AIChat and mdBook).
We publish our work instruction with chat capabilities using these tools.
We will use Linux's scheduler (cron) to keep everything up to date over time.

TODO: video of picture of success

use anywhere in the world (cloud, hybrid, on-premise)
use almost any llm in the world (including local llm)
Show example of how many releases from mdBook and aichat - newest models
keep your data safe (local if needed and protected by your intranet)
Three types of AI use (puts you in the top 90% or much better)
- AI to help create work instructions
- AI to help answer questions
- AI to help review review results and recommend changes
Script
- there is going to be a day when much of what we do is automated.
- systems will magically assist or perform much of the manual work.
- How do we tell it what to do?
- How do we start getting ready for that day - today?
- Asked another way, what is the minimum amount of effort I can do today to produce the most benefit in tomorrow's world
- This outline will thrust you ahead of 90% of other companies,
- We will combine best practice that align you with quality awards like the Baldrige award and start the augmentation/automation path?
- We will get your people ready for the rapidly approaching ai augmented workforce!

Here are the details discussed in the above video.

Before We Start

There are a couple of items you should note before we get started:

We assume you are publishing private work instructions for your organizations.
This example should not be exposed to the public internet for any appreciable period of time.

Linux

If you do not know how to create your first Linux server, here are some resources to help you get started:

Getting Started with Linux
Join the stack-academy - we will walk you through getting started
Ask a Linux friend to help.

Note that we will be using either Debian or Ubuntu on x86 (not ARM).

Prepare System

We need to install a few applications. The single easiest way to get your system prepared is using the chuboe system configurator.

Here is the process. You can simply copy all commands at once, paste and execute.

cd ~
sudo apt update
sudo apt install git -y
git clone https://github.com/chuboe/chuboe-system-configurator
cd chuboe-system-configurator
./init.sh

Install Sample Work Instruction

We need to install the sample work instruction repository.

Note: if you forked this repository, please update the below reference to chuckstack with the name of your fork.

cd ~
git clone http://github.com/chuckstack/ai-llm-operations-wi-chat/
cd ai-llm-operations-wi-chat/
./publish.sh init

Follow the steps that appear on your screen as the process ends to finish the initial configuration.

You should now be able to see your documents via a webpage by simply clicking on the indicated link.

Let's spend a moment discussing the airole-starter.md document. This text gets passed along side every prompt. This document gives you control over the user experience.

Soon we will discuss advanced topics related to balancing concise responses with the ability to ask for more details.

Chat is powerful; however, there are times when you need to simply search for something. Search is a powerful feature of this solution. We use mdBook to publish your work instructions and make them easily accessible to everyone in your organization.

Here are the ways to search:

ctrl+f (control + f key) to use the browser
s to search

Edit Online

Everyone in your organization should be able to easily contribute. The purpose of this section is to highlight just how easy it is.

Note that this example assumes many of the work instructions already exist. We will talk about how to get started creating your work instructions from scratch in a different section.

In the video we are using GitHub to host our work instructions. GitHub (and tools like it) provide online editing and advanced workflow to support change approval processes.

See the below FAQ for how to create a new work instruction from scratch.

Always Up to Date

The goal is to keep your published work instructions as up to date as is possible. Let's see how to automate updates.

Note: the rebuild section of the publish.sh script has git statements that need to be uncommented to support live updates. Look for ###change-me### next to the git sections.

Here are the details:

Configure git
Run manually
Run via cron

Note that your cron file might be named differently depending on what you are deploying. It will always start with 'cron-...'.

Talk to Any LLM

Quickly switch between almost any large language model (LLM) including local models. AIChat helps you focus on choosing the best model for the task making the testing and deploy almost trivial. AIChat not only powers this article, it is also one of the inspirations for creating the chuck-stack.

Control the User Experience

Talk like a parrot.
short then long (tell me more when requested)
Asking followup questions (include this?)

Review Responses and Improve

Rebuild process also reviews interactions and suggests improvements

TODO: video

Obsidian Workflow

Here is a video to help you visualize what its like to use Obsidian to manage work instructions.

TODO: video

Architecture Overview

Here are the tools we use to minimize effort and maximize agency.

Tools

We use Obsidian to help our work instruction creators write most documents in markdown.
- It includes AI assisted writing plugins.
- It includes git plugins to automatically track and publish changes to the website.
We use mdBook to convert our published markdown work instructions into a website.
We use AIChat to:
- embed the work instructions
- deliver a kind and patient 'chat' experience
- evaluate user interactions with the 'chat' tool
- propose improvements to work instructions
We use ttyd to expose AIChat and other terminal tools to our people.
We use nginx to host the mdBook website, reverse proxy ttyd, and terminate https/ssl connections.

ttyd

Next Steps

If you want help implementing these concepts in your organization, join the stack-academy. We will walk you though the process of:

deploying your first Obsidian + Git + website + chat solution in your organization - how to fork and deploy this example...
identifying and creating your first set of work instructions for a target role
teaching your content writers how to write the fewest words possible to achieve the desired result
soliciting feedback from everyone after the initial deployment

Frequently Asked Questions

Can I use Claude or other LLMs instead of OpenAI (ChatGPT)

Two considerations:

embeddings (used to take your work instructions and prepare them)
prompting (used to answer questions)

...

add video of success