Better Web Content, Better Chatbot Responses
Productivity
Mark Rosanes, AI Solutions Manager
AI Solutions
Tools
Copilot, Copilot Studio
Problem
Reviewing departmental websites for Shasta chatbot readiness was a manual, time-intensive process that required evaluating crawlability, accessibility, content quality, and compliance with chatbot and WCAG standards, making it difficult to scale across multiple departments.
Solution
The AI Solutions team built a Copilot Studio agent that automatically analyzes website content, identifies issues and improvement opportunities, and generates structured reports, enabling faster, more consistent reviews while helping departments improve content quality and chatbot performance.
It was clear from the outset that reviewing departmental websites as knowledge sources
for the Shasta chatbot would be a time-consuming process, reinforcing the need to improve content sources
to provide better context and clarity for effective generative responses. 
Each page needed to be checked for chatbot crawlability, accessibility, and content quality. Staff also had to make sure the content aligned with Gravyty chatbot content best practices and WCAG 2.2 accessibility guidelines. As more departments needed support, the manual review process became harder to scale consistently.
To improve the process, my team used Copilot and Copilot Studio to build an agent that reviews departmental websites supporting the Shasta chatbot.
The agent crawls selected URLs and evaluates the content for structure, accessibility, clarity, and usefulness as a chatbot knowledge source. It also identifies content gaps that may affect chatbot response quality or usability.
The agent produces a Word report that summarizes overall findings, themes, and recommended improvements, and includes a tabular, structured spreadsheet that lists each URL, identified issues, severity level, recommended changes, and the rationale behind each recommendation.
| Figure 1.1 – Report summary | Figure 1.2 - Structured spreadsheet output |
 |
 |
AI Solutions staff still review and validate the results before sharing them with
departments, but instead of starting from scratch, they now begin with a more organized,
AI-assisted review.
The result is a faster and more scalable process. Departments receive clearer, more actionable guidance to improve their web content. Shasta’s knowledge sources become easier for AI systems to crawl and interpret, which supports better chatbot responses.
This quick win shows how AI can strengthen work behind the scenes. By reducing repetitive review steps and improving consistency, AI helps staff focus on the higher-value work of validating recommendations, improving content, and supporting a better user experience.
The example, details, and outcomes described in this article were provided by the contributor. AI tools were used to help organize and draft the article, which was reviewed and validated by human staff to ensure accuracy, quality, and appropriate oversight.