BadApi" Reaktor challenge
RockPaperScissors "Motivation
Contains of this repository is webapp developed as a response to the "bad api" challenge, from API itself found here: BadApi endpoints.
Obstacles and measures
Application uses two interface points, history and live (ws://bad-api-assignment.reaktor.com/rps/live)
While live part could in essence be implemented with JavaScript/jQuery/Bootstrap by itself, but history part is bit more complicated.
Bigger problem is that server side history JSON isn't sent with proper headers (Access-Control-Allow-*), this is so called CORS problem. Problem could be solved by hacking user browser, in order to accept crosssite JSON, but I really don't think it is the purpose of assigment. More about that in next section.
Another problem with history is that it is divided into chunks.
live endpoint is bit troublematic, but as mentioned before, is solvable by mere jsonParser method, found in frontend/main.js
Also, quite huge problem, is devastatingly great count of JSON bits, and games in particular. JSONs usually about 1300-1400, and those JSONs contained about 840000 entries.
Huge amount of games could be handled by two methods, I see it fit, to fetch 1st JSON separately in both cases, but in one case - to download recursively with threads all JSONS, repack to new JSON (huge) and then send it to frontend.
I've, however, choose to leave in final version fetching of separate JSONs through middle-end proxy, and append them after loading, in sequence. Fetch-load-fetch-load...1400times...fetch-load.
Implementation
Well, main libraries to mention would be Flask from middle-end side, and jQuery,Bootstrap and DataTable from frontend side.
As previously mentioned, history endpoint wouldnt give JSON crosssite, so my solution is to implement middle end in Flask/Python.
It is also used to get recursively all JSONs first, then serve them to front-end. Also very slow, decided not to go with this as default choice, but method can be changed by changing methodRecursive in main.js
JSONs received are parsed and added to DataTable, which is slow, and could be done faster, by using vanilla table. 1st page is fetched, them id:scores table is shown.
After which, depending on recursiveMethod value, application either gets JSONs one by one through middle-end, and adding entries/games after each JSON is loaded.
recursiveMethod==true gives task of recursively downloading JSONs within middle-end, and then serving them to frontend. Adding rows is till slow, and this solution locks app badly, so I decided to leave recursiveMethod==false by default.
Usage
- Clone repository git clone https://gitlab.utu.fi/tivaju/bad_rps.git
- Change to application directory cd bad_rps
- Create new virtual enviroment venv for Flask/Python: python3 -m venv venv
- Activate venv: source venv/bin/activate
- Install Flask and other supporting libraries (while venv is active): pip3 install -r requirements.txt
- Run application: python3 main.py
- Open localhost in browser. (127.0.0.1:5000)
- Enjoy
Comments
Honestly said, I am not as experienced with JavaScript/jQuery, as with Python, therefore I might have made not really optimal choice of using DataTable.
Doing this task from zero ground, I'd do it without DataTable, but in vanilla table, as was initial plan, but got swayed by DataTable.
Rant: Didnt realize at first ?cursor=XXXXXX construction leading to much bigger load of data. for 1 JSON - webapp works flawlessly. Being mathematician and datascientist mostly, I am not that familiar with frontend. Threading in python, was also interesting experience, and probably done wrong
Otherwise, getting and parsing all JSONs is very unresponsive and slow, due to bit count of games. So I decided to go with slower, but more responsive version. One page at a time, can go very long until it gets all 1400 pages.
Ran for fun 127.0.0.1/history/some_id_from beginning. Cumulative JSON is about 16Mb, and it is quite-quite big. My, quite old desktop was able to recursively generate, and even somewhat load JSON to browser, but upon trying to use Firefoxes JSON parser, browser pretty much died.
From this, I really conclude that fetching everything at once is not optimal solution.
Also, I must mention, that this - is not bug on my behalf, I assume. (Ahti Virtanen vs Ahti Virtanen)
Some people have same names, so I guess it is possible that two Ahti Virtaset played a game