This post is about training a language generation model, that learns to write authentic looking 1-star reviews for video games.
I don’t even know where to begin with this game. It is utter trash. Terrible gameplay, terrible graphics, and terrible sound .The campaign was extremely boring and nobody really ever plays it for very long. The multiplayer mode is abysmal. The graphics are pixellated, the guns are inaccurate as heck, and to even bring it into the game you need to be a nfl star. If you are a regular joe you will be very dissapointed with this one.
GPT-2
GPT-2 is a model, developed by Open-AI researches that simply tries to predict the next word in a input sequence. The results were so good, that the first decided not to release the trained model, bit in the meantime they released it. More about their model can be read here.
We use the gpt-2-simple library, that as the name mentions simplifies the creation and training of language generation models. It can easily download pre-trained models and can be optimized very easily.
Video Review Dataset
The self-made dataset can be downloaded here. It is just a collection of reviews extracted from Amazon. Each line is a new review.
Create the Model
It takes hundreds of hours to train GPT-2 from scratch. Fortunately, we can simply download a pre-trained model, including the configuration. This saves us a lot of time and computation power.
There are multiple version for the model, that differ in size:
- 124M,500MB on disk.
- 335M, 1,5 GB on disk
- 774M
- 1558M
I recommend the 554M or 774M models, they both produce authentic results and are efficient to train.
We use the GPT-2-simple library, that as the name mentions simplifies the creation and training of language generation models. It can easily download pre-trained models and can be optimized very easily.
So far our model can already produce generic texts. But we want the generated output to look like negative game reviews, so we finetune the model with our small dataset of video game review.
To understand what the various parameters do and what exactly happens in the background, I recommend reading the documentation.
Generate Reviews
This is the fun part. We want to test the model and generate fake reviews.
This is the Output:
I have to say that it is a “relic” because it is so unbelievably hard to execute even the simplest of moves because the game requires 3-dimensional movement. The moves are painfully slow and require multiple attempts just to execute one move. In fact, the actual driving is so bad that it’s laughable. The car handling is abysmal. It’s almost comical, because even though it’s a motor vehicle, you cannot control the steering or the vehicle itself. It’s almost impossible to turn, which makes it impossible to maneuver and to stop. Finally, the game is extremely sexist, because even though the game claims to be a “game” for “girly” ladies, it is a gynocentric game that treats “girls” as though they are morons. If you’re a “real” girl, you’ll be appalled at how poorly this game is
Truly horrible, worst controls i have had for any game, bad storyline, and very very bad graphic… i give this game a zero out of 10… I was a ctually waiting in anticpation of this game when i heard of it but once i got it it was frustrating. i traded it in for 1/2 price the day after purchase. what was i ever thinking? As the saying goes “if you can’t beat it, you don’t try”. This game is as hard as nails. It has horrible gameplay, no good graphics, and just sucks. I give it a zero out of 10. This game was so icky that i did not even finish it.
This is one of those games that makes you wonder if there are any game designers at all who are capable of really delving into a game. Because if there are, it’s a game that requires a 3-D engine to run. So what happens is that the game designers come up with a lame excuse like “It’s a game of flying boats”.
