QSO Totals and WSJT-X imports

[G0DJA]

This may be two issues, but I discovered the second whilst trying to figure out the 1st, but I'll start a second topic if it will help.

I'm using Log4OM 2 v2.11.0.0
Windows 10 Home v1909 OsBuild 18363.1256

The first problem I discovered whilst trying to see how many contacts, in total, I have made using Phone, CW or Digital modes. I used the Filter option in QSO Manager to provide totals for the three different categories. These were as follows;

Phone = 3248
CW = 3894
Digital = 2059

Total = 9201 However, on the main screen the total number of logged contacts was 9221, a discrepancy of 20 contacts.

So, I started to try and find out why there were missing contacts. I checked through the 'modes' in the various categories to see if, at some point in the past, I had used some strange mode that did not appear in the lists of modes counted in the three categories. I could not find anything by just looking through a few contacts so I exported a CSV file of some basic data (Date, Callsign, Mode) and loaded it into a spreadsheet. After sorting by 'Ascending' mode type and scrolling down to see if I could find 20 contacts with some strange mode I didn't find any mode that did not appear in the lists of modes used (I found a post on here telling me where to find the file that contains the modes used by Log4OM) so I did some more digging.

I decided to sort all my contacts by callsign, to see if I'd managed to log some strange call that was not valid, and I found that, for a large number of contacts, mainly FT8 contacts, I had duplicates on the same day (at this point I had not looked at the times for the contact...) so I started to check through these duplicates.

After about a days work I had found over 500 duplicated contacts, mainly FT8, where the date was the same but the time logged was just a few seconds different. Usually 15 seconds different, which of course is one period for FT8 in either TX or RX.

So, I worked through my database using QSO Manager, inputting each callsign and deleting the second logged entry. Had I known this was going to involve removing so many I might have asked for help earlier... Anyway, it's done now and I'll watch the imports from WSJT-X in future to make sure I don't get duplicate entries. That showed up another slight issue. After searching for about 30 callsigns, and deleting one of the duplicates, the search seemed to slow down significantly and I found I had to close Log4OM and then restart it. Then do about 30 more deletions and restart Log4OM again. I guess the search function was never designed for someone wanting to find and delete over 500 records manually?

Anyway, back to the main issue. I went back to see how many CW, Phone, Digital contacts I had made. Now the totals were;

Phone = 3242
CW = 3894
Digital = 1507

Total = 8643

Looking at the total contacts logged Log4OM is showing a total of 8663 contacts, so I'm still 20 contacts short...

Any ideas what these missing 20 might be please?

Thanks - Dave (G0DJA)

[G4POP]

This may be two issues, but I discovered the second whilst trying to figure out the 1st, but I'll start a second topic if it will help.

I'm using Log4OM 2 v2.11.0.0
Windows 10 Home v1909 OsBuild 18363.1256

The first problem I discovered whilst trying to see how many contacts, in total, I have made using Phone, CW or Digital modes. I used the Filter option in QSO Manager to provide totals for the three different categories. These were as follows;

Phone = 3248
CW = 3894
Digital = 2059

Total = 9201 However, on the main screen the total number of logged contacts was 9221, a discrepancy of 20 contacts.

So, I started to try and find out why there were missing contacts. I checked through the 'modes' in the various categories to see if, at some point in the past, I had used some strange mode that did not appear in the lists of modes counted in the three categories. I could not find anything by just looking through a few contacts so I exported a CSV file of some basic data (Date, Callsign, Mode) and loaded it into a spreadsheet. After sorting by 'Ascending' mode type and scrolling down to see if I could find 20 contacts with some strange mode I didn't find any mode that did not appear in the lists of modes used (I found a post on here telling me where to find the file that contains the modes used by Log4OM) so I did some more digging.

I decided to sort all my contacts by callsign, to see if I'd managed to log some strange call that was not valid, and I found that, for a large number of contacts, mainly FT8 contacts, I had duplicates on the same day (at this point I had not looked at the times for the contact...) so I started to check through these duplicates.

After about a days work I had found over 500 duplicated contacts, mainly FT8, where the date was the same but the time logged was just a few seconds different. Usually 15 seconds different, which of course is one period for FT8 in either TX or RX.

So, I worked through my database using QSO Manager, inputting each callsign and deleting the second logged entry. Had I known this was going to involve removing so many I might have asked for help earlier... Anyway, it's done now and I'll watch the imports from WSJT-X in future to make sure I don't get duplicate entries.

At some point you must have been employing both the UDP and the ADIF monitor systems for WSJT in Log4OM, if you do that duplicates are created because WSJT logs the END time in its ADI log file whereas it sends the start time by UDP

You can easily cure this by exporting a complete log from Log4OM as an ADIF file and reimporting it into an empty database at the import select 'Duplicates threshold' to 30 or 60 seconds which will then consider any smaller time difference as a dupe.

Untitled.jpg (77.83 KiB) Viewed 2188 times

That showed up another slight issue. After searching for about 30 callsigns, and deleting one of the duplicates, the search seemed to slow down significantly and I found I had to close Log4OM and then restart it. Then do about 30 more deletions and restart Log4OM again. I guess the search function was never designed for someone wanting to find and delete over 500 records manually?

Never experienced this and I have spent hours deleting qso's from users logs, like thousands. I remember having to help G0JMZ sort out his log after logger32 screwed it up and I had to delete over twenty thousand QSO's

Anyway, back to the main issue. I went back to see how many CW, Phone, Digital contacts I had made. Now the totals were;

Phone = 3242
CW = 3894
Digital = 1507

Total = 8643

Looking at the total contacts logged Log4OM is showing a total of 8663 contacts, so I'm still 20 contacts short...

Any ideas what these missing 20 might be please?

Not sure what you have done but my best suggestion is to create a new database and import a recent Backup ADIF from before you did any deletions but user the duplicates threshold described earlier, that will ensure a dupe free database

[G0DJA]

Anyway, back to the main issue. I went back to see how many CW, Phone, Digital contacts I had made. Now the totals were;

Phone = 3242
CW = 3894
Digital = 1507

Total = 8643

Looking at the total contacts logged Log4OM is showing a total of 8663 contacts, so I'm still 20 contacts short...

Any ideas what these missing 20 might be please?

Not sure what you have done but my best suggestion is to create a new database and import a recent Backup ADIF from before you did any deletions but user the duplicates threshold described earlier, that will ensure a dupe free database

Thanks for the reply,

I'll keep in mind the UDP/ADIF issue and will import a few FT8 contacts when I've made some more, and see what happens.

However, the missing 20 contacts don't seem to be related to the FT8 duplicates issue, as I still have 20 that are 'missing' between the totals of the three Phone/CW/Digital scores and the total number of contacts that are reported to be in the log...

Cheers - Dave (G0DJA)

[G4POP]

Check the recorded modes or perhaps lack of same

[G0DJA]

Check the recorded modes or perhaps lack of same

I decided to export the full data set, and create a new database, as you suggested.

Whilst it was importing the ADIF file that I'd created from the previous copy of the database I noticed a number of notes saying "Received mode not recognized PACKET" from back in January 1989.

Once it finishes importing I'll see if I can access these records and change the mode to "PKT" which I think it will recognize?

Cheers - Dave (G0DJA)

[G4POP]

That will do it because Packet is not, and as far as I recall has never been, a recognised ADIF mode field


Not a Log4OM error so moved topic to 'User support'

[G0DJA]

I've managed to get the totals to add up now.

I changed to "PACKET" mode to "PKT" and that got me to within 7 of the full total but I had also noticed an error that "JT8" was not a recognized mode either. To cure this I had another idea...

I opened the file 'modelist' in the folder AppData\Roaming\Log4OM2 and added a line for the mode "JT8" between the digital modes JT6M and JT9. That cures the final 7 missing contacts so now the three Phone/CW/Digital totals add up the same number as the total contacts recorded in the database.

Cheers - Dave (G0DJA)

[G0DJA]

I had another thought.

Will there be an updated modelist file available when the RC versions of WSJT-X become a full release, probably later this month? Because there are two new modes called 'FST4' and 'FST4W' which I can't see listed in the copy of modelist that I have.

Cheers - Dave (G0DJA)

[G4POP]

Did you update the mode list via the log4om update facility in the settings menu?

[G0DJA]

Did you update the mode list via the log4om update facility in the settings menu?

I have done in the past, but I'll do it again in a minute and see if the new modes are added. I don't think JT8 will as it was removed from WSJT quite a long time ago now.

Cheers - Dave (G0DJA)