Simplified Search Engine
Wow, I finished making this project today. I’m really happy with it! I can’t really believe that I made it. There was quite a bit of technology involved, and then also the normal writing code and Mimir shenanigans.
My friend suggested the idea upon hearing my expense tracker idea — my intention is basically to have the students build a project that can build upon different data structures I’m teaching. That’s the requirement, and initially I was super excited about my expense tracker because it fit what I wanted. Then this search engine idea bloomed and I ended up doing it instead.
We’re going to progress from lists to 2-D lists to dictionaries to complex dictionaries (aka dictionaries in which the value is not just a value but a list or a dictionary) to perhaps sets, stacks, and queues, so I wanted to find a project that can evolve to fit all of these data structures. I’m grabbing articles and their metadata to put into these data structures, and the program will have the user search for a keyword and provide information accordingly. The simplified search engine isn’t really going to be an engine though; it’s really just going to search through a data structure for items with relevant keywords. There’s also no ranking system, although I did think about incorporating it somehow.
I used BigQuery’s public Wikipedia dataset to grab articles and their metadata with article names containing a certain set of words so that during the first part of this project when the code is searching through for article names, there will actually be the same words that span across all the articles (I only grabbed about 100 articles). When I did this initially, I didn’t think about the overlapping words in titles and the search project code didn’t come up with any results exceeding one article title.
So after I grabbed the data, which didn’t include the actual article, I called Wikipedia’s API to actually get the article, and then I parsed it to see what words are actually in the article. When I did that, I was really, really, really thankful and grateful for teaching in Python because that makes my life infinitely easier and faster. It’s just so much faster to code in Python than anything else.
After that, I had all the data, processed in a way that would be provided to my students. That file was pretty fun to write because it’s not normally code I would need to write, and I haven’t written any code like that for this class so far.
I then figured out what functions my students would write, and I had a pretty big headache over that because I didn’t want to call the functions myself in the file I provide. I wanted them to practice calling functions, so I wanted to make them write their own “main” function. But, that would be too much work, so I initially set on giving them the skeleton. That seemed too confusing though and today, I actually had code for them to call the functions they write but I eventually commented it out and wrote in the instructions to uncomment it after they finish writing the function. I guess in hindsight, I didn’t really achieve what I wanted. I do also want them to integration test it though and in order to do that, they should have access to the code that they’re supposedly integration testing.
I finished writing the code today including unit and integration tests, as well as some of the unit tests to test student functions on Mimir. I’m not sure how I’m going to test their integration test yet, but I’ll probably just make sure it runs.
I also spent time writing the instructions document. It ended up being eight whole pages. Holy cow! I was so surprised, and I’m really glad I like writing so it didn’t feel like a chore.
I feel very accomplished and happy with this. I think I pretty much just did what could be considered a side project. This is pretty cool! I’ve never made a project before (the first project done by the students was borrowed), and it’s really fun. I’m a bit scared of how it might turn out, but I’m hoping it will be okay.
