Tibia Fandom Web Scrapper

Description

Simple web scrapper fetching data about creatures from Tibia -> the MMO game from Tibia Fandom.

Reading basic bestiary information about inputed creatures by it's URL.

Used technology

• Python3

Dependencies

• beautifulsoup4

Commands

Available commands:

• creature - scraps basic bestiary information, main functionality of the scrapper
• list - scraps list of urls leading to creatures on Tibia Fandom, can be used for further bestiary scrapping

Creature scrapping example

Input URLs (or scrap it with list command) of creatures to scrap, e.g:

https://tibia.fandom.com/wiki/Acid_Blob
https://tibia.fandom.com/wiki/Badger
https://tibia.fandom.com/wiki/Chicken

App will scrap data about these monsters and save output to json file:

[
    {
        "charmPoints": 25,
        "creatureClass": "Slime",
        "creatureClassification": "Blobs",
        "experience": 250,
        "health": 250,
        "name": "Acid Blob",
        "resistances": {
            "Death": 100,
            "Earth": 100,
            "Energy": -10,
            "Fire": -10,
            "Ice": 20
        },
        "version": "8.40"
    },
    {
        "charmPoints": 5,
        "creatureClass": "Mammal",
        "creatureClassification": "Mustelids",
        "experience": 5,
        "health": 23,
        "name": "Badger",
        "resistances": null,
        "version": "7.4"
    },
    {
        "charmPoints": 5,
        "creatureClass": "Bird",
        "creatureClassification": "Birds",
        "experience": 0,
        "health": 15,
        "name": "Chicken",
        "resistances": null,
        "version": "7.5"
    }
]

How To Use

1. Clone main branch of the project on your device
2. Install required dependencies (check dependencies)
3. Create in.txt file in project directory
4. Place creatures URLs to scrap in the in.txt file (check example) or scrap the whole list with list command
5. Choose command you want to run (check commands)
6. Run py Main.py [command]
7. Check your output in out.json file or list.txt based on which command you are using

Changelog

• 1.1

  • list command -> scrapping list of Creatures urls

• 1.0

  • creature command -> scrapping data about Creatures:
    • name
    • experience
    • health
    • class
    • classification
    • charm points
    • resistances
    • version