{"id":186582,"date":"2025-05-05T10:22:41","date_gmt":"2025-05-05T14:22:41","guid":{"rendered":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/?p=186582"},"modified":"2026-05-04T10:43:47","modified_gmt":"2026-05-04T14:43:47","slug":"paychex-2","status":"publish","type":"post","link":"https:\/\/www.hajim.rochester.edu\/senior-design-day\/paychex-2\/","title":{"rendered":"Paychex"},"content":{"rendered":"\n

Project Title:<\/strong> An AI-Powered Regulatory Chatbot for Labor Legislation
Team Members<\/strong>: Ethan Leung, Carol Li, Astha Singh, Keming Zhang
Sponsor:<\/strong> Paychex
Affiliation<\/strong>: Goergen Institute for Data Science, University of Rochester<\/p>\n\n\n

INTRODUCTION<\/h2>\n\n\n

In today\u2019s fast-evolving regulatory landscape, businesses must constantly adapt to changes in federal and state legislation to ensure compliance in areas such as payroll, taxation, and human resources. Paychex, a leading provider of integrated HR solutions, faces the challenge of maintaining up-to-date regulatory information to effectively support over 740,000 clients. This capstone project focused on an <\/span>AI-powered chatbot<\/b> designed to assist Paychex in <\/span>delivering accurate and timely regulatory guidance<\/b>.<\/span><\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

https:\/\/news.cornell.edu\/stories\/2024\/07\/research-decision-making-mystery-ai-chatbots<\/a><\/p>\n\n\n\n

OBJECTIVES<\/h2>\n\n\n\n

Our work culminates in a scalable, automated system that improves access to complex legal information, supporting efficient and dependable client service. By combining advanced AI methods with robust data management, the system aims to provide Paychex with reliable, up-to-date information, which enhances operational efficiency and ensures adherence to complex regulatory requirements.<\/p>\n\n\n\n

    \n
  1. Analyze and prepare data for model integration<\/li>\n\n\n\n
  2. Build and optimize the vector database<\/li>\n\n\n\n
  3. Develop the retrieval-augmented generation agent graph<\/li>\n\n\n\n
  4. Integrate the chatbot with data and graph functions<\/li>\n<\/ol>\n\n\n\n

    DATA DESCRIPTION<\/strong><\/h2>\n\n\n\n

    The original dataset comprises regulatory information related to labor laws at both the federal and state levels<\/strong>. The dataset initially contained 1,124 entries<\/strong>, with each row representing a distinct regulation, accompanied by a brief description and a hyperlink to the official source. One entry was removed due to a missing URL, resulting in a final dataset of 1,123 complete records for analysis.<\/p>\n\n\n\n

    There are three types of data sources\u2014Web Resource, PDF, and HTML<\/strong>\u2014each scraped using distinct techniques. A total of 1036 links were successfully scraped<\/strong>, while 87 errors remain. All extracted text was compiled into the dataset.<\/p>\n\n\n\n

    \"\"<\/figure>\n\n\n\n

    Figure 1: Distribution of File Types<\/em><\/p>\n\n\n\n

    \"\"<\/figure>\n\n\n\n

    Figure 2: Distribution of Errors<\/em><\/p>\n\n\n\n

    METHODS<\/strong><\/h2>\n\n\n\n

    Feature Engineering<\/strong><\/h3>\n\n\n\n

    Procedure based on the Description column:<\/p>\n\n\n\n

      \n
    • Added \u201cCategory<\/strong>\u201d column based on keywords in the description: paid_sick_leave, childcare_regulations, wage_minimum_industry, holiday_pay_provisions\uff0cwork_hour_regulations, employment_statutes, fair_labor_standards\uff0cemployee_leave_benefits.<\/li>\n\n\n\n
    • Added a Jurisdiction<\/strong> column to categorize each row as either Federal or State.<\/li>\n\n\n\n
    • Added State and State Code<\/strong> columns to classify all State-level regulations according to the 50 U.S. states.<\/li>\n<\/ul>\n\n\n\n

      Among the eight categorized themes, paid_sick_leave<\/em> accounts for the highest number of regulations (255), followed by childcare_regulations<\/em> (238) and wage_minimum_industry<\/em> (216). In contrast, categories such as fair_labor_standards<\/em> (64) and employee_leave_benefits<\/em> (44) are represented less frequently. This distribution highlights the relative prominence of certain labor regulation themes, particularly those related to paid leave and childcare, within the dataset.<\/p>\n\n\n\n

      \"\"<\/figure>\n\n\n\n

      Figure 3: Distribution of Regulation Categories<\/em><\/p>\n\n\n\n

      Another feature added during the engineering process was the Jurisdiction<\/em> column, which classifies each regulation as either Federal or State in scope. A figure was created to show the distribution of Jurisdiction. We can observe that there are 911 state-related regulations and 212 federal-level regulations. The number of state regulations is highly outweighs the number of federal laws.<\/p>\n\n\n\n

      \"\"<\/figure>\n\n\n\n

      Figure 4: Distribution of Jurisdiction<\/em><\/p>\n\n\n\n

      Furthermore, two supporting columns\u2014State<\/em> and State Code<\/em>\u2014were introduced to facilitate the classification of all state-level regulations according to the 50 U.S. states. Analysis of these columns reveals that the majority of state regulations in the dataset originate from a subset of states, including Maryland (MD), Minnesota (MN), New York (NY), California (CA), Montana (MT), Arizona (AZ), New Jersey (NJ), Massachusetts (MA), New Mexico (NM), and Maine (ME). This distribution may reflect differences in regulatory activity or data availability across states.<\/p>\n\n\n\n

      \"\"<\/figure>\n\n\n\n

      Figure 5: Distribution of Number of Laws by State<\/em><\/p>\n\n\n\n

      FAISS Vector Database<\/strong><\/h3>\n\n\n\n

      A brief steps as follows:<\/p>\n\n\n\n

        \n
      1. Database  2. Embedding   3. FAISS Index  4. Indexing   5. TF-IDF  6. Searching<\/li>\n<\/ol>\n\n\n\n

        We used FAISS (Facebook AI Similarity Search) to build a fast, efficient search system that matches user queries to relevant labor laws. By converting legal text into vector embeddings using models like BERT, the chatbot can quickly find semantically similar content. We also incorporated TF-IDF scoring to enhance result accuracy and relevance.<\/p>\n\n\n\n

        Hybrid Search<\/strong><\/h3>\n\n\n\n

        To improve precision, we combined semantic vector search with traditional keyword matching in a hybrid approach. This ensures that the chatbot not only understands the meaning behind a query but also captures important legal terms. By ranking results based on both methods, the chatbot delivers more relevant and legally accurate answers.<\/p>\n\n\n\n

        LangGraph<\/strong><\/h3>\n\n\n\n

        We used LangGraph to manage the chatbot’s multi-step reasoning, including searching, filtering, and generating responses. This structure allows the system to fall back on a controlled Google Search if no internal data matches a query, ensuring reliability. Prompts guide each AI agent\u2019s role, enabling smooth and accurate interactions.<\/p>\n\n\n\n

        RESULTS<\/strong><\/h2>\n\n\n\n

        The project resulted in the creation of three models that improved upon each other. The workflow of the chatbot is pictured below, along with a model comparison of each model\u2019s performance in a few common scenarios. Observably, the first version was the fastest but gave too succinct responses. The second sacrificed speed for answer quality, but fell short in hyper-specific queries. The final sees the same speed decrease, but it works well in all scenarios.<\/p>\n\n\n\n

        \"\"<\/figure>\n\n\n\n

        Figure 6: Model Comparison of Each Model\u2019s Performance<\/em><\/p>\n\n\n\n

        \"\"<\/figure>\n\n\n\n

        Figure 7: LangGraph Workflow Version 3<\/em><\/p>\n\n\n\n

        DEMO<\/strong><\/h2>\n\n\n\n
        \n