Hybrid Context Retrieval Augmented Generation Pipeline: LLM-Augmented Knowledge Graphs and Vector Database for Accreditation Reporting Assistance

In higher education, accreditation is a quality assurance process, where an institution demonstrates a commitmentto delivering high quality programs and services to their students. For business schools nationally andinternationally the Association to Advance Collegiate Schools of Business (AACSB) accreditation is the goldstandard. For a business school to receive and subsequently maintain accreditation, the school must undertake arigorous, time consuming reporting and peer review process, to demonstrate alignment with the AACSBStandards. For this project we create a hybrid context retrieval augmented generation pipeline that can assist in thedocumentation alignment and reporting process necessary for accreditation. We implement both a vector databaseand knowledge graph, as knowledge stores containing both institutional data and AACSB Standard data. Theoutput of the pipeline can be used by institution stakeholders to build their accreditation report, dually groundedby the context from the knowledge stores. To develop our knowledge graphs we utilized both a manualconstruction process as well as an ‘LLM Augmented Knowledge Graph’ approach. We evaluated the pipelineusing the RAGAs framework and observed optimal performance on answer relevancy and answer correctnessmetrics.