Every other week, the FastRuby.io newsletter opens a new window brings a curated list of the best Ruby and Rails articles, tutorials, and news to your inbox.

Our engineering team collects links to interesting articles and our marketing team curates them, writes a summary for each article, and creates the newsletter. This process is quite manual, and involves some back and forth to ensure summaries are accurate, engaging, and relevant to our audience.

To make if more efficient, we have developed an AI assistant that helps us curate articles and generate the summaries for the newsletter.

Why an AI Assistant?

We wanted a tool that could reduce the repetitive parts of the workflow without taking away the human touch that is essential for effective communication. Summarizing a dozen articles every other week can be tedious and time-consuming, but it is necessary. We still want summaries that sound like us and highlight the right things. Hence the AI assistant.

The AI assistant leverages a large language model (LLM) to analyze the content of the articles, extract key points, and generate concise summaries. This helps our marketing team save some time and focus on the areas of the newsletter that require human creativity and judgment.

The Stack

We wanted something that was easy to build, could be set up quickly and could also be used by our marketing team. This is an internal tool, and we just wanted something quick that works.

We chose to build the AI assistant using:

• Sinatra: to create a simple interface for our marketing team to interact with the AI assistant.
• pgvector: to store and query vector embeddings of the article summaries.
• Langchain.rb: to handle the interaction with the embedding model, the LLM, and to manage the workflow.

For the embeddings, we used OpenAI’s ada-002 model, which is well-suited for generating high-quality embeddings for text. For the LLM, we used OpenAI’s gpt-40 model.

How It Works

To make it easy for our team to suggest links, we created a simple Slack integration that works through a Slack command. When a team member suggests a link, the AI assistant:

• Fetches the article’s HTML content.
• Extracts the title and main content using nokogiri (a Ruby HTML parser).
• Does some minimal cleaning of the content to remove unnecessary elements.
• Embeds the content using the ada-002 model to create a vector representation.
• Stores the title, content, and vector in a PostgreSQL database using pgvector.
• Triggers the summary generation process.

We’ll walk through the summary generation process in detail.

Summary Generation

Immediately after the article is added, the AI assistant generates a summary using the gpt-40 model. First, it retrieves three examples from our database of previously generated summaries using similarity search with pgvector. Performing a cosine similarity search on our articles table with pgvector is quite easy:

def fetch_examples(article)
  examples = article.nearest_neighbors(:embedding, distance: "cosine").limit(3)
  examples.map(:summary)
end

Here, article is an instance of the Article model, which has a pgvector column called embedding. The nearest_neighbors method retrieves the three most similar articles based on their embeddings.

Next, the AI assistant generates a summary using a generate and review strategy. It first generates a draft summary based on the article content and the examples retrieved. Then, it reviews the draft against the examples and a set of instructions to ensure it aligns with our style and tone. If it does, it approves the draft. If it doesn’t, it provides feedback to be used to refine the draft.

The first draft is generated using a prompt with the following structure:

<<~PROMPT
  [Context: What kind of assistant is this?]
    
  [Context: What will the assistant be looking at?]

  [Task]

  1. Instruction number 1
  2. Instruction number 2
  3. Instruction number 3

  [Call to action: What should the assistant do?]

  **Examples of past summaries:**
  #{examples.map { |ex| "- #{ex.strip}" }.join("\n")}

  **Blog Post:**

  *Title:* #{title.strip}

  *Content:*
  #{content.strip}

  Return your response in this JSON format. Return ONLY the JSON object.
  {
    "title": "...",
    "summary": "..."
  }
PROMPT

After the draft is generated, the AI assistant reviews it using a prompt structured like this:

<<~PROMPT
  You are a critical editor, review the snippet below:

  **Title:**
  #{title}

  **Article Content:**
  #{content}

  **Summary:**
  #{summary}

  Compare this snippet to the tone, length and style of these examples:
  #{examples.map { |ex| "- #{ex.strip}" }.join("\n")}

  Is it:
  - Characteristic number 1
  - Characteristic number 2

  Does it:
  - Question number 1
  - Question number 2

  If the snippet is accurate and acceptable, respond ONLY with:
  {"approved": true}

  If it needs edits, respond ONLY with:
  {"approved": false, "feedback": "...", "revised_summary": "..."}
PROMPT

The generate function looks like this:

def generate_summary(url, parsed_blog, max_attempts: 3)
    raise "URL is required" if url.nil? || url.empty?

    examples = fetch_examples(parsed_blog[:content])
    
    # Generate the initial summary
    summary = generate(parsed_blog[:title], parsed_blog[:content], examples)
    
    # Review the generated summary
    revised_summary = review(parsed_blog[:title], parsed_blog[:content], summary, examples, max_attempts)
    revised_summary[:summary]
  end

Where the generate and review methods handle the interaction with the LLM using Langchain.rb.

def generate(title, content, examples)
    prompt = prompts.generate_snippet(title, content, examples)
    summary = client.chat(prompt, system_prompt: prompts.system)
    raise "Incomplete snippet: #{summary}" unless summary[:title] && summary[:summary]

    summary
end

def review(title, content, summary, examples, max_attempts)
  attempt = 1
  while attempt < max_attempts
    prompt = prompts.critic(title, content, summary[:snippet], examples)
    review = client.chat(prompt, system_prompt: prompts.system)

    return summary if review[:approved]

    raise "Critic failed to provide a revised snippet" unless review[:revised_summary]

    summary[:summary] = review[:revised_summary]
    attempt += 1
    break if attempt > max_attempts

  end
  summary
end

This process allows the AI assistant to generate summaries that are not only accurate but also aligned with our brand’s voice and style.

Summary Re-Generation

If the AI assistant generates a summary that is not quite suited for the newsletter, it can be easily re-generated by the marketing team.

The team can simply click a button in the interface and add their feedback for the model to consider, and that will trigger the summary regeneration process. Optionally, they can also change the temperature of the LLM to make the output more or less creative.

Regenerating a summary is similar to the initial generation, but it includes the feedback provided by the marketing team:

<<~PROMPT
  [Context: What kind of assistant is this?]
 
  You are correcting a snippet that has been suggested and rejected. When creating a snippet, you must always consider the following:

  1. Instruction number 1
  2. Instruction number 2

  [Task]

  [Additional instructions on how to handle the feedback provided.]

  **Feedback:**
  #{feedback.strip}

  **Blog Post:**

  *Title:* #{title.strip}

  *Content:*
  #{content.strip}

  **Previous Snippet:**
  #{snippet.strip}

  Return your response in this JSON format. Return ONLY the JSON object.
  {
    "title": "...",
    "snippet": "..."
  }
PROMPT

Regeneration does not include the review step, as the feedback is already provided by the marketing team:

def regenerate(title, content, snippet, feedback, temperature)
    prompt = prompts.regenerate_snippet(title, content, snippet, feedback)
    if temperature
      temperature = temperature.to_f / 10
      client.temperature = temperature
    end
    revised_snippet = client.chat(prompt, system_prompt: prompts.system)
    raise "Incomplete snippet: #{revised_snippet}" unless revised_snippet[:title] && revised_snippet[:snippet]

    revised_snippet
  end

Our marketing team can then just copy the summary to use in the newsletter content, or tweak it further if needed.

Conclusion

The AI assistant we built for the FastRuby.io newsletter has helped streamline our workflow, allowing our marketing team to focus on the creative aspects of curation while automating the repetitive tasks of gathering and summarizing links.

Through a mix of LLM-powered functionality, a simple interface, and a Slack integration, we have been able to create a tool that saves our marketing team a significant amount of operational time.

Want to know how we can help you leverage AI for your business? Talk to us today! opens a new window .