Page 1 of 5

Resetting RaidStats Highscore/Ranking lists(?)

PostPosted: 20 Mar 2016, 00:04
by Dilatazu
Hey all

The RaidStats database is currently 10 times larger than it was thought to ever become.
This is causing a lot of issues for the entire site due to the amount of data that needs to be reloaded and updated all the time.

I began working on migrating to a new database system back in beginning of the new year after i received some donations(thanks a lot everyone!) that really motivated me to work hard to solve the stability issues. My thought was that i was going to do the same migration with RaidStats after i was done with the RealmPlayers part. Doing the realmplayers migration was a lot of work though and this has really caused me to reconsider if it is worth doing it for RaidStats at all. I still dont enjoy working on the project in the current state. So i could not get myself to do the work for it, knowing the long road ahead and all the issues is just too much.

I have debated whether i should remake RaidStats from scratch for a long time, this would solve a lot of issues. But i have thought about it for such a long time and still have not got around to actually do it. Even when i eventually start i will have no idea when i will be able to be done with it or if i ever will get done with it.

In the meantime the ranking and highscore database is becoming bigger and bigger. One day i have a feeling it will just completely stop working or get corrupted every day.

So the question is: Should i reset the raidstats highscore/ranking lists completely now? Or should we wait for the day it blows up and i have no choice but to do it either way?
Feel free to give some feedback of the situation in the thread and discuss what you think is the best decision, or if you have a better idea.

I also want to be clear that the situation since september has not really changed. RealmPlayers is pretty much still only alive because of donations and the fact that i do not want to abandon people. So thanks a lot to all the project supporters! And if anyone wants to get involved in the coding, please contact me!

Re: Resetting RaidStats Highscore/Ranking lists(?)

PostPosted: 20 Mar 2016, 01:11
by noodle
does this include instance clear times?

Re: Resetting RaidStats Highscore/Ranking lists(?)

PostPosted: 20 Mar 2016, 01:27
by Dilatazu
noodle wrote:does this include instance clear times?


Yes this would unfortunately reset all instance clear times aswell.

The only data left on the site would be all raids listed and their raidoverview and fightoverview pages.

Re: Resetting RaidStats Highscore/Ranking lists(?)

PostPosted: 20 Mar 2016, 01:32
by Juddy
Dilatazu wrote:
noodle wrote:does this include instance clear times?


Yes this would unfortunately reset all instance clear times aswell.

The only data left on the site would be all raids listed and their raidoverview and fightoverview pages.


Given that it would appear to be a more desirable solution to remove data from now dead realms as well as Nostalrius data which was recorded prior to Blackwing Lair release - any raid in, and before, August 2015.

Re: Resetting RaidStats Highscore/Ranking lists(?)

PostPosted: 20 Mar 2016, 01:37
by Soyoen
Hey dila, is there any possibility to nuke all shitdata and keeping the good (best raids) for each of the guilds/players/servers?

In other words all shit data that isn't included in the records for the server/guild/player? Would probably be alot of unwanted work, just wanted to know if there is any easy possibility that the records could be preserved?




Otherwise you could always do a clean wipe every 6-12months and log the best instance clears/dps records in a seperate hall of fame page. Log as in a screendump or something similar ;)

Anyways
Best of regards
Soyo

Re: Resetting RaidStats Highscore/Ranking lists(?)

PostPosted: 20 Mar 2016, 01:42
by Dilatazu
Juddy wrote:Given that it would appear to be a more desirable solution to remove data from now dead realms as well as Nostalrius data which was recorded prior to Blackwing Lair release - any raid in, and before, August 2015.


All data recorded before Nostalrius released is only 1/5 of the database. So it would not really make much difference, in a month or two we would be back to exactly same situation as now.

Soyoen wrote:Hey dila, is there any possibility to nuke all shitdata and keeping the good (best raids) for each of the guilds/players/servers?
In other words all shit data that isn't included in the records for the server/guild/player? Would probably be alot of unwanted work, just wanted to know if there is any easy possibility that the records could be preserved?

It might be a solution to nuke all data older than X months or not up on the highscore lists. But it would probably require a lot of work.

Soyoen wrote:Otherwise you could always do a clean wipe every 6-12months and log the best instance clears/dps records in a seperate hall of fame page. Log as in a screendump or something similar ;)
I like this idea, but it would also require some work.

Re: Resetting RaidStats Highscore/Ranking lists(?)

PostPosted: 20 Mar 2016, 01:44
by Juddy

All data recorded before Nostalrius released is only 1/5 of the database. So it would not really make much difference, in a month or two we would be back to exactly same situation as now.



All data prior to September 2015 is 1/5 of the data?

Re: Resetting RaidStats Highscore/Ranking lists(?)

PostPosted: 20 Mar 2016, 01:45
by Dilatazu
Juddy wrote:

All data recorded before Nostalrius released is only 1/5 of the database. So it would not really make much difference, in a month or two we would be back to exactly same situation as now.



All data prior to September 2015 is 1/5 of the database?


All data prior to september 2015 is 2/5 of the database.

Re: Resetting RaidStats Highscore/Ranking lists(?)

PostPosted: 20 Mar 2016, 01:46
by noodle
Every solution other than nuking all the data seems like it would be a lot of work, and if you aren't open to doing "a lot of work" why did you even ask for peoples suggestions?

The best solution that preserves what people care about most is simply nuking the data prior to Blackwing Lairs release on nostalrius.

Also, you can nuke all Zul'Gurub raids because they are completely irrelevant regardless of when they were recorded, the same as Onyxia and maybe even Molten Core.

What % of the data is from Onyxia and Zul'Gurub?

Re: Resetting RaidStats Highscore/Ranking lists(?)

PostPosted: 20 Mar 2016, 01:53
by Soyoen
noodle wrote:Also, you can nuke all Zul'Gurub raids because they are completely irrelevant regardless of when they were recorded, the same as Onyxia and maybe even Molten Core.

What % of the data is from Onyxia and Zul'Gurub?

If ZG is deemed irrelevant then you can put it on a nuking schedule if possible, aka nuke its data every ~2months due to the amount of times it can be done each week.