Giggle GPT Joke Search Engine – Django ORM & Data Saving (3/6)

by

Welcome back to part 3, where we will be saving our data to the database using the Django ORM.

🎓 Full 6-video course with downloadable PDF certificates: Giggle – Creating a Joke Search Engine from Scratch with ChatGPT

Before we get started, as always, make sure your virtual environment is running.

Linux(bash):

$ source venv/Scripts/activate

Windows:

$ venv/Scripts/activate (uses the.bat file)

You should see (venv) in your terminal before or on the current line.

So our search engine is working, that’s all nice and well, but our jokes are gone as soon as we refresh the page or search a new query. We will now use the Django ORM to actually save our data to the database so we can have persistent data.

Quick recap

We discussed the Django ORM in part 1 but let’s have a brief recap.

SQL databases normally work with SQL queries. Django comes loaded with the ORM though, which allows us to define objects and execute methods on them. Django will use the Object Relational Mapper to execute the appropriate SQL queries for us.

One advantage of this is that later if we want to switch to a different production-grade database instead of SQLite which we usually use for basic development, we can just change the database in our settings.py file and not worry about changing any of the queries, because they are all generated by Django for us.

Naturally, a database needs a schema, to tell it what kinds of data we will be storing and what the table will look like. Remember back in part 1 where we defined a model?

Open models.py again. A model is the schema for a Django object that the ORM will use to manipulate our database.

We only have to manipulate the object and run its methods and Django will handle the rest for us!

Finally remember that whenever we write new models or change existing ones, we need to use the two-step process to get our changes from Django into the real database. You don’t have to run these now.

python manage.py makemigrations
python manage.py migrate

The Django Admin Panel

Next, we will make a superuser. This will make it easier to browse the database for those of you that do not have SQLite browser installed.

The Django Admin interface is automatically generated based on the models defined in your application’s models.py file. It will allow us to perform CRUD operations (create, read, update, delete) on our website before writing the code to do so.

To use the Django Admin interface, you first need to create a superuser account.

python manage.py createsuperuser

If you plan to actually deploy this project anywhere later (and keep access to the admin panel open) make sure you pick a very strong password. I’m just gonna use admin and admin for now.

Make sure your server is running.

python manage.py runserver

And then go to /admin. If you log in you should see the admin panel. We have ‘Groups‘ and ‘Users‘ here, which both come by default with Django, but our Jokes are not showing, even though we ran ‘makemigrations‘ and ‘migrate‘ back in tutorial part 1.

We need to register them for them to show up in the admin panel.

Open admin.py in the main giggle folder

from django.contrib import admin
from .models import Joke

admin.site.register(Joke)

We import from dot models, which means this folder’s models file, our Joke model. We use the admin object already imported for us to register our model named Joke in the Django admin panel.

Make sure to save your changes as always. If we refresh our Django admin we now have the Jokes show up. Django took the name of our model Joke and pasted an 's' onto it. If we open it the list of jokes is empty because we have not saved any jokes so far.

Saving Jokes to the Database

Now let’s go to views.py in the giggle folder, and near the top of views.py add:

from .models import Joke

This is the same import we used a moment ago in our other file. We need to import the Joke model so we can manipulate it and its associated database data from our views.py file.

Now we need to make a new Joke object. We can create a new ‘instance’ of Joke data by simply calling the Joke model and catching the return in a variable.

Insert the following two lines in your giggle_search function below the context['response'] = response line:

new_joke_object = Joke(query=query, response=response)
new_joke_object.save()

Note that we first pass in the query and then the response, these names are the names we defined ourselves in the Joke model inside models.py, and therefore the names we must use.

We did not pass in a date-time even though we specified the created_at field in our Joke model since we set Django to automatically use the current date-time for us.

Now we have a Python object of our Joke model but populated with the data, which we caught in the variable named ‘new_joke_object‘. At this point, it only exists in memory. We need to call the .save() method on our object for Python to use the ORM, generating and running the SQL required to save the data to the appropriate table in our database.

Your giggle_search function should now look like this:

def giggle_search(request):
    context = {}
    if request.method == 'POST':
        query = request.POST.get('query')
        response = get_giggle_result(query)
        context['query'] = query.capitalize()
        context['response'] = response
        new_joke_object = Joke(query=query, response=response)
        new_joke_object.save() 

    return render(request, './giggle/giggle_search.html', context)

Now go ahead and load the main page on your Development server and try a query. Then switch to the admin interface at /admin again and click on Jokes. You should now see something there. The name is pretty bad "Joke object x", but our data is there in the database!

Joke object (1)
    Query:      ice cream
    Response:   Why did the ice cream truck break down? Because it had a Rocky Road!

String Representation using the str() Method

Let’s fix the name. To do this we go back to our models.py.

Notice that Joke is a class. To give each object a representative name, we must define the __str__ dunder (double underscore) method. This is not a Django-specific thing, but vanilla Python.

The __str__ dunder method defines a string representation for the host object/class if the print() function is called on it. When you run a print() statement on an object Python really just calls its __str__ method to get a string representation that can be printed.

class Joke(models.Model):
    query = models.CharField(max_length=50)
    response = models.TextField()
    created_at = models.DateTimeField(
        verbose_name='Created at',
        auto_now_add=True,
    )

    def __str__(self):
        return self.query

Inside the class of Joke, we already had, we define a method double underscore str double underscore, which takes self as an argument, so that we have access to the properties we defined above.

Now we simply return whatever we want to use as the string representation, using self.query to access the query defined above.

Refresh the Django admin, and you will now see:

ice cream
    Query:      ice cream
    Response:   Why did the ice cream truck break down? Because it had a Rocky Road!

Let’s go ahead and add the date as well. Change your dunder string method to:

def __str__(self):
    return f"{self.query} - {self.created_at}"

We use a simple f-string to insert the query, a dash, and then the creation date. If you refresh the admin panel again you will see that the timestamp has been added.

ice cream - 2023-06-04 01:43:07.023548+00:00
    Query:      ice cream
    Response:   Why did the ice cream truck break down? Because it had a Rocky Road!

We won’t go deeper into the date-time formatting as we’ll only be using the admin panel for our own convenience here, and the end user will never see this.

Try another search query in your giggle search engine page and then go back to the admin panel to see another object has appeared under the Jokes heading.

JOKE
grass - 2023-06-04 01:59:34.939893+00:00
ice cream - 2023-06-04 01:43:07.023548+00:00

Retrieving Jokes from the Database

Our data is automatically being stored in the database each time we perform a search. Now we need to retrieve this data from the database and allow the end user to see it inside the real webpage.

Let’s start off easy by retrieving just the previous two jokes before the current one and displaying them below the current joke.

Go back to our views.py and let’s call up the previous jokes from the database. Do this outside the conditional if request.method == 'POST': block.

previous_jokes = Joke.objects.all()
context['old_jokes'] = previous_jokes

This again uses the Django ORM. It tells Django to find the corresponding data in the database and retrieve ALL data for our Jokes.

We want all objects associated with the Joke model. This is basically the SQL query SELECT * FROM jokes.

Next, we pass this data to the giggle_search.html template using the context like we always have when passing data from our view function to the template.

Your giggle search function now looks something like this:

def giggle_search(request):
    context = {}
    previous_jokes = Joke.objects.all()
    context['old_jokes'] = previous_jokes
    if request.method == 'POST':
        query = request.POST.get('query')
        response = get_giggle_result(query)
        context['query'] = query.capitalize()
        context['response'] = response
        new_joke_object = Joke(query=query, response=response)
        new_joke_object.save() 

    return render(request, './giggle/giggle_search.html', context)

Go to the giggle_search.html template inside our giggle/templates/giggle folder to catch this data. We will be editing the div with the class of ‘results‘ again and leaving the rest of the HTML untouched.

<div class="results">
    {% if query %}
        <h3>{{ query }}</h3>
        <p>{{ response }}</p>
    {% endif %}
    {% if old_jokes %}
        {% for joke in old_jokes %} 
            <h3>{{ joke.query }}</h3>
            <p>{{ joke.response }}</p>
        {% endfor %}
    {% endif %}
</div>

In here, we start to see the real power of the template engine.

The top part is the same, if there is a variable named query in the context dictionary, show the last query and its response. After that, we test if there is a variable named old_jokes in the context dictionary.

Now you’ll see a for joke in old_jokes template syntax for loop.

Note that it is very similar to Python in that we get to name the smaller part whatever we want. We could also have said, for automobile in old_jokes, or for alligator in old_jokes and it would work just the same.

For each joke in old_jokes, we print the joke query and the joke response.

Notice we simply use the names we defined in our Joke model inside models.py to access the data again, isn’t that simple and clean!

Finally, make sure to close both the for loop and the if loop using {% endfor %} and {% endif %}. Loops must be closed just like conditionals and blocks when using template syntax. The indentation here is only for our own reading convenience but does not affect the execution as it would in Python code.

If we now reload the page we will see our old jokes below the newly generated one! Try submitting a new query, searching for anything you feel like.

Bears
Why did the bear cross the road? To get to the honey on the other side!

ice cream
Why did the ice cream truck break down? Because it had a Rocky Road!

grass
Why did the grass go to the doctor? Because it was feeling lawnly.

bears
Why did the bear cross the road? To get to the honey on the other side!

Refining our Database Query

Notice we still have some issues.

First of all, we have our last query showing twice, because we add it to the database and then also read all jokes from the same database again, passing the new query result to our template separately and then again through the old_jokes data.

Second, the list of old jokes below the new query has the oldest joke first, instead of the one previous to the current one. I would ideally like to see bears (the newest), and then grass (the second newest), and then ice cream (the oldest).

Let’s go back to views.py and fix this.

We would normally use SQL queries to extract, filter out and order the needed data from the database. Luckily the Django ORM has equivalents or similar functions for all the SQL queries that you might need. Let’s fix the order of the old jokes first.

In views.py change the previous_jokes = Joke.objects.all() line as follows:

previous_jokes = Joke.objects.all().order_by('-created_at')

We chain on the order_by method and then specify the field to order on, in this case the ‘created_at‘ field that we defined when we made the model and instructed Django to automatically set to the current date-time. If you look closely you will see a - in front of the created_at name, this dash specifies Django that we want to sort in reverse order, giving us the newest first, instead of the oldest.

Make sure to save so the server will serve up the newest page, reload, and voila, the order goes from newest to older downwards.

This page is going to get very long as time goes on, so let’s limit it to displaying only the 2 most recent old jokes. Edit the same previous_jokes = Joke.objects.all().order_by('-created_at') line again:

previous_jokes = Joke.objects.all().order_by('-created_at')[:2]

If you reload the page now you will only see 2 old jokes maximum.

Notice we are using simple Python slicing syntax to just slice the first two entries off. You might think we are wasting resources by calling up the entire database table and then slicing off only the first two entries but Django actually is clever enough to consider the slice at the end as part of the query.

Django does not get the entire database and then slice a small part off but reads the entire line of code and only gets what we need.

This type of object and the data it returns is called a Queryset in Django, and what you must understand is that Querysets are lazy. What that means is that we can chain filters behind the Joke part all day long, I could add 10 more filters and conditions, and Django will never execute a single query before getting to the end. No database resources are wasted.

Now search for a new query using your giggle search engine and you will notice there is one problem remaining. The newest joke you just searched is still being retrieved from the database as well and thus displaying twice.

Banana
Why did the banana go to the doctor? Because it wasn't peeling well.

banana
Why did the banana go to the doctor? Because it wasn't peeling well.

golf players
Why did the golfer wear two pairs of pants? In case he got a hole in one!

We could solve this in different ways, but as we’ll be moving the historical results to a separate page in the future we’ll keep it simple and use the slice syntax.

def giggle_search(request):
    context = {}
    if request.method == 'GET':
        previous_jokes = Joke.objects.all().order_by('-created_at')[:2]
        context['old_jokes'] = previous_jokes
    if request.method == 'POST':
        previous_jokes = Joke.objects.all().order_by('-created_at')[1:3]
        context['old_jokes'] = previous_jokes
        query = request.POST.get('query')
        response = get_giggle_result(query)
        context['query'] = query.capitalize()
        context['response'] = response
        new_joke_object = Joke(query=query, response=response)
        new_joke_object.save() 

    return render(request, './giggle/giggle_search.html', context)

We start the function and define an empty context as always. Then if the method is a GET request, which means the user just loaded the page without posting a query, we simply want to display the two newest jokes on the page below the search bar using a slice of [:2], starting on index zero and ending on 2.

If the request method is POST, we simply use a slice of [1:3] starting on index 1 and ending on 3 to cut off the newly created item, preventing it from displaying twice. Everything below is the same as the function we already had.

Now if you first load the page, you will see the two most recent historical results.

a duck
Why did the duck cross the road? To prove he wasn't chicken!

oil and water
Why did the oil and water break up? Because they were just not meant to be together.

And if you add a new query you will see it and the two most recent searches before it.

Smells fishy
Why did the fish blush? Because it saw the ocean's bottom.

a duck
Why did the duck cross the road? To prove he wasn't chicken!

oil and water
Why did the oil and water break up? Because they were just not meant to be together.

Styling

Let’s fix the styling a little bit to emphasize the most important recent result. Go to giggle_search.html in our giggle/templates/giggle folder. Wrap each of the two (h3) query (p) response blocks inside of a div with a class of joke and joke_old respectively.

{% if query %}
    <div class="joke">
        <h3>{{ query }}</h3>
        <p>{{ response }}</p>
    </div>
{% endif %}
{% if old_jokes %}
    {% for joke in old_jokes %} 
        <div class="joke_old">
            <h3>{{ joke.query }}</h3>
            <p>{{ joke.response }}</p>
        </div>
    {% endfor %}
{% endif %}

Then go to your styles.css file in the static folder, and add the following selectors to the bottom of the file.

.joke_old {
    color: #888;
}

.joke_old:last-child {
    color: #AAA;
}

We give the joke_old class a different font color and then select the joke_old class with the pseudo-selector of last-child, selecting only the second old_joke and giving it a slightly lighter color still. Now if you reload the page you will see the new styling.

Finally, we add a quick CSS selector to fix the capitalization of the old jokes’ titles.

.joke_old:first-letter {
    text-transform: uppercase;
}

This selects the joke_old class and then the first-letter of the content, applying a transformation of uppercase.

We will be moving our old jokes to a separate history page in part 4, turning this into a multi-page application. But before we think about adding new pages we need to discuss links in Django so that we can link them together.

Let’s go back to our urls.py in the giggle folder. Make sure not to open the one in the project folder instead, remembering we have 2 urls.py files.

from django.urls import path
from . import views

urlpatterns = [
    path('', views.giggle_search, name='giggle_search'),
]

Linking from within templates

Notice we only have a single route right now, for an empty path.

The view it calls is the giggle_search function in views.py we have been working on so far. As we hinted at in an earlier part of the tutorial the name='giggle_search' part is the namespace. We use this name to create links to this route from within our templates. This name is basically our way to access the router from within our templates.

Say we have a hypothetical route as follows (do not put this one in your code):

path('information/', views.information, name='information'),

Now we could create links in our HTML using an href="/information" and this would work just fine. But say we want to change the path of information to something else later. (Again, do not add this one to your code.)

path('amazing_information/', views.information, name='information'),

Now all our HTML links to href="/information" will be broken. But notice the name='information' part has not changed. If we had used the namespace as our link instead then nothing would have broken even if we want to change the name of the relative path. This is why we use names to link.

Let’s go to giggle_search.html and use the logo image as a link back to the main page (even though we only have one page right now).

...
<header>
    <a href="{% url 'giggle_search' %}">
        <img class="img-fluid" src="{% static 'giggle.png' %}" alt="Giggle logo" />
    </a>
</header>
...

We just wrapped the img element in an a (anchor) tag and in the href we use template syntax {% %} within which we specify we want to link in a url and then simply provide the namespace we want to link to in quotation marks.

It’s that simple, Django will handle the rest for us.

Now if you save and reload your main page and click on the logo it should reload the page you are already on, as we are linking to the same page.

Checking User Input

Before we continue to the next part we have one more issue to fix.

So far we have taken the user’s input and used it as is. We should never trust users to cooperate and we should never trust user input. Django actually escapes HTML in templates by default, so the user could not post HTML tags in their query and then run JavaScript from there to embed an XSS attack into your webpage.

Our database is also decently protected from SQL injection by having the additional Django ORM layer in between, so we’re pretty safe there.

The user can, however, easily type input that is way longer than 50 characters even though we specified a maximum of 50 characters in our models.py file. If we want to limit something we must enforce it. Let’s go to our views.py file.

Find the first line of code which reads:

from django.shortcuts import render

and add another import called redirect, like so:

from django.shortcuts import render, redirect

We can use this redirect function inside our views.py view functions to throw a redirect from one view function to another, from one route to another.

And now in your giggle_search function, add a simple test to see if the query is not too long.

if len(query) > 50:
    return redirect('giggle_search')

If the query is longer than 50 characters, we return with a redirect back to the giggle_search page without executing the API call or saving anything to the database.

The redirect function also uses the namespace we used just a moment ago, so we can simply input the name we want to redirect to like ‘giggle_search‘. Make sure to place your if statement in the right position like this:

def giggle_search(request):
    context = {}
    if request.method == 'GET':
        previous_jokes = Joke.objects.all().order_by('-created_at')[:2]
        context['old_jokes'] = previous_jokes
    if request.method == 'POST':
        previous_jokes = Joke.objects.all().order_by('-created_at')[1:3]
        context['old_jokes'] = previous_jokes
        query = request.POST.get('query')
        if len(query) > 50:
            return redirect('giggle_search')
        response = get_giggle_result(query)
        context['query'] = query.capitalize()
        context['response'] = response
        new_joke_object = Joke(query=query, response=response)
        new_joke_object.save() 

    return render(request, './giggle/giggle_search.html', context)

Now open your page and type in a super long query over 50 characters, and you should notice that nothing happens, you are simply redirected to the main search page without any query being executed or a result showing up.

For any user who is not maliciously trying to break our page and send us too many characters on purpose, we should just make sure they cannot accidentally type too many characters in the first place.

Go to giggle_search.html in the giggle/templates/giggle folder and add the attribute maxlength="50" to the existing input field.

<input type="text" name="query" class="form-control" placeholder="Input topic" maxlength="50" />

Now the HTML input box will limit the input to 50. If you try this out on your page you will see that after 50 characters you simply cannot type any more. Note you should obviously never rely on this HTML attribute alone as anyone who wants to can easily edit it, but it provides a good user experience for all normal users.

That’s it for part 3. I hope to see you soon in part 4 where we will implement a second route and page into our project and also make it possible to delete jokes.