AI Agent for resume shortlisting

Today we are going learn how to build a resume shortlisting AI agent using ChatGPT and LangChain, that involves several steps as follows.

1. Define the Scope and Requirements

• Objective: Automate resume shortlisting based on job descriptions.
• Input: Resumes (in plain text or parsed format) and job descriptions.
• Output: A ranked list of resumes with relevance scores or recommendations (e.g., shortlisting or rejecting).
• Key Features: Text parsing, keyword matching, semantic understanding, and explainability.

2. Set Up Your Development Environment

Install necessary libraries

pip install langchain openai pdfplumber nltk

• LangChain: Framework for building AI workflows.
• OpenAI API: Access to ChatGPT for natural language processing.
• PDF Parsing (optional): Tools like pdfplumber for extracting text from resumes.
• NLTK or Spacy: For preprocessing and keyword extraction.

3. Parse and Preprocess Resumes

Parsing Resumes:

• Use pdfplumber or similar libraries to extract text from resume files.
• For structured formats (like JSON or XML), extract relevant sections like skills, experience, and education.

Preprocessing:

• Tokenize text.
• Remove stopwords.
• Extract entities like job titles, years of experience, and skills using libraries like Spacy.

import pdfplumber
def parse_resume(file_path):
 with pdfplumber.open(file_path) as pdf:
 text = ""
 for page in pdf.pages:
 text += page.extract_text()
 return text.strip()

LangChain can help you create a pipeline for shortlisting:

Load Resumes:

• Store parsed resumes in a document store (e.g., in-memory or vector databases like Pinecone or FAISS).

Embed Resumes and Job Descriptions:

• Use OpenAI or Hugging Face embeddings to represent resumes and job descriptions as vectors.

from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import FAISS

def create_embeddings(resumes, job_description):
    embeddings = OpenAIEmbeddings()
    
    # Embed resumes
    vectors = FAISS.from_texts(resumes, embeddings)
    
    # Embed job description
    job_vector = embeddings.embed_query(job_description)
    
    return vectors, job_vector

Compare Relevance:

• Compute similarity scores between job description embeddings and resume embeddings.

from langchain.vectorstores import FAISS
vector_store = FAISS.from_texts([resume_text], embeddings)
similar_docs = vector_store.similarity_search(job_description, k=5)

def find_similar_resumes(vectors, job_vector, k=5):
    similar_docs = vectors.similarity_search_with_score(job_vector, k=k)
    return similar_docs

from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain
from langchain.chat_models import ChatOpenAI

def evaluate_resumes_with_gpt(similar_resumes, job_description):
    gpt_model = ChatOpenAI(temperature=0)  # Use ChatGPT
    prompt = PromptTemplate(
        input_variables=["resume", "job_description"],
        template="Evaluate the following resume against the job description. Provide a relevance score (0-10) and a short explanation.\n\nJob Description:\n{job_description}\n\nResume:\n{resume}"
    )
    chain = LLMChain(llm=gpt_model, prompt=prompt)
    
    recommendations = []
    for resume_text, score in similar_resumes:
        response = chain.run({"resume": resume_text, "job_description": job_description})
        recommendations.append({"resume": resume_text, "gpt_response": response, "similarity_score": score})
    
    return recommendations

5. Generate Recommendations Using ChatGPT

Use ChatGPT for semantic analysis and ranking:

• Prompt Engineering: Feed the job description and each resume to ChatGPT for evaluation.

from langchain.prompts import PromptTemplate
prompt = PromptTemplate(
    input_variables=["resume", "job_description"],
    template="Evaluate the resume against the following job description. Provide a relevance score (0-10) and a brief explanation.\n\nJob Description:\n{job_description}\n\nResume:\n{resume}"
)

• Execution:

from langchain.chains import LLMChain
chain = LLMChain(llm=openai_gpt, prompt=prompt)
response = chain.run({"resume": resume_text, "job_description": jd_text})

6. Flask API for Deployment

from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route('/shortlist', methods=['POST'])
def shortlist_resumes():
    # Get job description
    job_description = request.form['job_description']
    
    # Upload and parse resumes
    resumes = []
    for resume in request.files.getlist('resumes'):
        parsed_text = parse_resume(resume)
        resumes.append(parsed_text)
    
    # Generate embeddings and find similar resumes
    vectors, job_vector = create_embeddings(resumes, job_description)
    similar_resumes = find_similar_resumes(vectors, job_vector)
    
    # Evaluate with GPT
    recommendations = evaluate_resumes_with_gpt(similar_resumes, job_description)
    
    return jsonify(recommendations)

if __name__ == '__main__':
    app.run(debug=True)

7. UI Design

Here’s a simple UI design using HTML, CSS, and JavaScript to accept a job description and a set of resumes for uploading. The UI integrates with the Flask API described earlier.

• Job Description Input: A textarea for entering the job description.
• Resume Upload: An input[type="file"] element allows multiple PDF resumes to be uploaded.
• Submit Button: Sends the job description and resumes to the backend API (/shortlist).
• JavaScript: Handles form submission via fetch to send data to the Flask backend and displays results dynamically.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Resume Shortlisting</title>
    <style>
        body {
            font-family: Arial, sans-serif;
            margin: 0;
            padding: 0;
            background-color: #f4f4f9;
        }
        .container {
            max-width: 800px;
            margin: 50px auto;
            padding: 20px;
            background: #fff;
            border-radius: 8px;
            box-shadow: 0 4px 8px rgba(0, 0, 0, 0.1);
        }
        h1 {
            text-align: center;
            color: #333;
        }
        label {
            font-weight: bold;
            margin-top: 10px;
            display: block;
            color: #555;
        }
        textarea, input[type="file"], button {
            width: 100%;
            margin-top: 10px;
            padding: 10px;
            border: 1px solid #ccc;
            border-radius: 4px;
            font-size: 16px;
        }
        button {
            background-color: #007BFF;
            color: white;
            border: none;
            cursor: pointer;
        }
        button:hover {
            background-color: #0056b3;
        }
        .output {
            margin-top: 20px;
            padding: 10px;
            background: #e9f5ff;
            border-left: 4px solid #007BFF;
        }
        .error {
            background: #fbe9e7;
            border-left: 4px solid #d9534f;
        }
    </style>
</head>
<body>
    <div class="container">
        <h1>Resume Shortlisting</h1>
        <form id="resumeForm">
            <label for="jobDescription">Job Description</label>
            <textarea id="jobDescription" rows="6" placeholder="Paste the job description here..." required></textarea>

            <label for="resumes">Upload Resumes (PDF format)</label>
            <input type="file" id="resumes" multiple accept=".pdf" required>

            <button type="submit">Submit</button>
        </form>

        <div id="output" class="output" style="display:none;"></div>
    </div>

    <script>
        document.getElementById('resumeForm').addEventListener('submit', async function (event) {
            event.preventDefault();

            const jobDescription = document.getElementById('jobDescription').value;
            const resumes = document.getElementById('resumes').files;

            if (!jobDescription || resumes.length === 0) {
                alert("Please fill out the job description and upload resumes.");
                return;
            }

            const formData = new FormData();
            formData.append('job_description', jobDescription);
            for (let i = 0; i < resumes.length; i++) {
                formData.append('resumes', resumes[i]);
            }

            try {
                const response = await fetch('http://127.0.0.1:5000/shortlist', {
                    method: 'POST',
                    body: formData,
                });

                const data = await response.json();

                const outputDiv = document.getElementById('output');
                outputDiv.style.display = "block";
                outputDiv.innerHTML = `<h3>Shortlisting Results</h3><pre>${JSON.stringify(data, null, 2)}</pre>`;
            } catch (error) {
                const outputDiv = document.getElementById('output');
                outputDiv.style.display = "block";
                outputDiv.className = "output error";
                outputDiv.innerHTML = `<h3>Error</h3><p>There was an error processing your request. Please try again later.</p>`;
            }
        });
    </script>
</body>
</html>

8. How to Integrate with Flask Backend

Run the Flask Backend: Save the app.py file and run:

python app.py

• This starts the Flask server at http://127.0.0.1:5000/.
• Open the Frontend: Save the HTML file (frontend.html) and open it in a browser.

Workflow:

• Enter the job description in the provided field.
• Upload one or more resumes (PDF format).
• Click Submit to send data to the Flask backend.

Results:

• The backend processes the job description and resumes.
• The frontend displays a JSON response with the shortlisting results.

9. Further Enhancements

• Add validation for file types and job description length.
• Display a progress indicator while uploading files.
• Improve the output display with a table or visual ranking.