It take longer now to log a qso

V2 error reports
9k2hn
Advanced Class
Posts: 41
Joined: 03 Aug 2014, 09:33

Re: It take longer now to log a qso

Post by 9k2hn »

The manual says if you have more than 100k qso it is better to use MYSQL, I do have 526k qso.

Is moving to sqlite is good option?
User avatar
G4POP
Log4OM Alpha Team
Posts: 10815
Joined: 21 Jan 2013, 14:55
Location: Burnham on Crouch, Essex UK

Re: It take longer now to log a qso

Post by G4POP »

Wow!

No with a log like that stick with Mysql but you need to manage the Mysql database properly and I am not an expert on fine tuning MSQL
73 Terry G4POP
9k2hn
Advanced Class
Posts: 41
Joined: 03 Aug 2014, 09:33

Re: It take longer now to log a qso

Post by 9k2hn »

While importing my log in patches I noticed that when I reached 200k qso all was great and fast but after that, it started to get very slow.

I hope someone on this forum can guide me on how to fine-tune MySQL to be faster (I'm open for suggestion)
9k2hn
Advanced Class
Posts: 41
Joined: 03 Aug 2014, 09:33

Re: It take longer now to log a qso

Post by 9k2hn »

But something for sure in the latest release of 2.31.0.0 made it much slower than before
User avatar
IW3HMH
Site Admin
Posts: 2927
Joined: 21 Jan 2013, 14:20
Location: Quarto d'Altino - Venezia (ITA)
Contact:

Re: It take longer now to log a qso

Post by IW3HMH »

may i ask an ADIF of your log? (downloadable from somewhere if possible)
Seems really huge and, surely, a good benchmark for me to improve current implementation...
i will try adding some indexes and check how they performs on mysql, then i will add them to default installation and i will send you scripts

Thanks
Daniele Pistollato - IW3HMH
9k2hn
Advanced Class
Posts: 41
Joined: 03 Aug 2014, 09:33

Re: It take longer now to log a qso

Post by 9k2hn »

Email is sent to you with a link to my full logbook around 1 GB in size
User avatar
IV3IPS
Novice Class
Posts: 22
Joined: 15 Aug 2019, 07:41
Contact:

Re: It take longer now to log a qso

Post by IV3IPS »

I had the exact same problem. The database was in the PC Google Drive folder. I did a test by moving the database to the c:\documents folder.... solved the problem ! Now it saves qso's in reasonable time. I hope I've been helpful.

73 Stefano IV3IPS.
9k2hn
Advanced Class
Posts: 41
Joined: 03 Aug 2014, 09:33

Re: It take longer now to log a qso

Post by 9k2hn »

I Have moved now to SQLite and it is working quite fast, my question is there any future issues with a logbook with 500k qso and SQLite?
User avatar
G4POP
Log4OM Alpha Team
Posts: 10815
Joined: 21 Jan 2013, 14:55
Location: Burnham on Crouch, Essex UK

Re: It take longer now to log a qso

Post by G4POP »

In theory no but see how you get on.

I am convinced there has been some change in MySQL that we dont know about causing odd issues for some users
73 Terry G4POP
w9mdb
Old Man
Posts: 437
Joined: 13 Jul 2014, 12:05

Re: It take longer now to log a qso

Post by w9mdb »

Google Drive syncs take around 20 minutes or more.
So everything probably works OK for a while and then slows down for a while.
So I'd think it would be every other QSO or such.
Google Drive probably locks the file to upload it (so it doesn't change under foot) and that's when mysql will wait until the lock is released.
MySQL will not lock the file as that's the whole point of a database like that to allow multiple access.
If I were you I'd just run a periodic job to copy your database to Google Drive. No problem then on locking.
Post Reply