The Tech Buffet #20: How To deploy a Cloud Function That Summarizes Youtube Videos

A practical guide on industrializing a serverless LLM app with an infrastructure-as-code (IaC) framework

Hello everyone 👋 Ahmed here. I’m the editor of The Tech Buffet, a newsletter that delivers practical insights on shipping production-ready ML applications.

In this issue, you will learn how to automate the deployment of a serverless Cloud Function on Google Cloud.

This application will do something that I always wanted to prototype: summarizing YouTube videos 📹.

With the powerful LLMs that are currently available, the scalable cloud architectures, and the efficient developer tools, bringing this idea to life is now possible.

So here’s what we’ll cover to build this project:

• A Brief Introduction to Cloud Functions

• How to develop and test a Cloud Function locally

• How to design the infrastructure on GCP and deploy the Cloud Function on it

In the last step, we will use Python code only to provision the infrastructure and deploy the Cloud Function. We won’t be manipulating the GCP UI or using an Infrastructure as Code (IaC) tool that requires another syntax.

If you’re interested in DevOps or MLOps and how to ship ML applications at scale, it might be useful to stick to the end 👇

Let’s have a look.

---

What are Cloud Functions?

If you come from the AWS world, Cloud Functions are the equivalent of Lambda..

Cloud Functions is a Google Cloud service that allows you to run your code in the cloud with no servers or containers to manage and in a scalable and cost-efficient way.

That’s why Cloud Functions are called serverless.

With Cloud Functions, the developer experience is simple: you write and test your code locally then send it to Google Cloud to deploy and handle the operational infrastructure.

The second great feature of Cloud Functions is the cost: you only pay for what you use. Specifically, you’re billed for your function’s execution time, metered to 100 milliseconds. And when your function is idle, you pay nothing!

Cloud Functions are primarily designed to process short-lived, event-based actions triggered from other systems such as Cloud Storage or PubSub.

For example, you can build a Cloud Function that gets triggered when a file is saved under a bucket or when a row is inserted in BigQuery.

I also see Cloud Functions being used for sending notifications or triggering other systems.

---

Develop and Test the Cloud Function locally

In this small project, the Cloud Function we’ll build will act as a REST API. It’ll receive user inputs consisting of YouTube URLs and send back the summary.

Before going through the deployment on GCP, let’s first build and test this function locally.

To summarize YouTube videos, we will need these Python libraries:

• youtube-transcript-api: to extract transcripts from a YouTube video

• langchain and langchain-openai: to interact with OpenAI LLMs and generate the summary

• python-dotenv: to load OpenAI credentials as environment variables

• functions-framework: a FaaS (Function as a Service) framework for writing portable Python functions and testing them locally

The simplest thing to get started is to create a virtual environment:

python -m venv ./venv
source venv/bin/activate
pip install -r function/requirements.txt

Then, the function’s code will be defined in function/main.py:

Here’s the main function that’ll summarize the video (for more details check the remaining code in the repo):

→ It first extracts the video transcript and the title, initializes an LLM, builds a prompt, and sends it to the LLM for summarization.

def summarize_youtube_video(url, additional_instructions):
    transcript = get_youtube_video_transcript(url)
    title = get_youtube_video_title(url)
    llm = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))
    prompt = get_prompt(title, transcript, additional_instructions)
    summary = llm.predict(prompt)
    data = {
        "url": url,
        "title": title,
        "summary": summary,
        "transcript": transcript,
    }
    return data

To wrap this logic in a REST API, we’ll use the functions_framework package. This allows us to define the Cloud Function handler (or entry point) that will be called once an HTTP request is sent:

@functions_framework.http
def main(request: flask.Request):
    if request.method == "POST":
        url = request.form.get("url")
        additional_instructions = request.form.get("instructions")
        data = summarize_youtube_video(url, additional_instructions)
        return flask.jsonify(data)
    else:
        return "Method Not Allowed", 400

The full Cloud Function code is available here.

To test the Cloud Function locally, run the following command:

functions-framework --target=main --source=function/main.py --debug

Running the command will start a local web server. So let’s open Postman to send it some requests:

Let’s send this video that discusses the differences between Terraform and Pulumi:

By filling in the form data with the URL and hitting the Send button, we get the following result: a JSON response with the following keys:

• summary

• title

• transcript (the full text)

• url

Here’s a closer look at the summary:

Terraform and Pulumi are two tools that are often compared in the world of DevOps. Terraform is known for its consistency in defining state, while Pulumi offers a more fluid approach using imperative programming languages such as Python and C#. However, this can also lead to potential issues when working in a team or switching organizations.\n\nTerraform, on the other hand, uses HCL (Hashicorp Configuration Language) to define state in a more limited and consistent manner. This allows for easier collaboration and a more secure way of managing state files. Additionally, Terraform is not a SaaS product, meaning your state is not automatically pushed to a remote network. This can be a concern when handling sensitive information.\n\nOverall, the speaker strongly prefers Terraform over Pulumi due to its simplicity and consistency. They also mention that Terraform has a community of nearly 2,000 members who can offer support and answer any questions. In the end, the speaker urges viewers to use Terraform in its simplest form and avoid overcomplicating things.

Pretty cool, right? Imagine doing this for 30-minute world news reports every day. This would be a huge time saving!

If you reproduce this tutorial, you can play with the Postman client, try different videos, and even add a custom instruction to the LLM.

In the following example, you can ask the LLM to answer in French.

The Cloud Function is tested locally.

We’re now ready to deploy.

---

Provision an infrastructure for the Cloud Function

If you asked me to deploy a Cloud Function a few years ago, I would happily log into the GCP console, put the source code into a bucket, create the Cloud Function from the UI, add a secret in Google Secret Manager, connect it to the app, and hit the deploy button.

If we were to perform these steps every time we make a change to the source code, this would be:

• tedious

• prone to errors

• untraceable in a codebase

• difficult to collaborate on

Enter Pulumi 🚀