Why is the system so slow? Must we add more SSD drives?

Why are all the meters not calling in by 8am? It seems to be struggling.

Why are the energy totals and other registers not lining up with the customer records from the previous system?

The beginning of that is that your acquisition threads were set to 200, and your database output threads were set to 4.

Better to set the acquisition threads to 80 and the database output threads to 20.

Also what would help, is to set the following property in your Server.prp file:
DBPOOL=40
for that to be in effect, the JVM process must restart. 

Currently we're also still working on the Meter Account Summary meter to be more efficient, so currently there is a bottleneck when all the meters call that meter account to update and they are serialized, so pls start with a simpler system, esp if you want to read in a lot of data from the new meters, e.g. months of data. So until you gain back control of your Acquisition system side of things, pls change to a simpler system, and get that working first, and then incrementally add changes.

I think it is important to switch off Events for all the meters until we have stabilized
because on the ACE6000 and SL7000, the only way to read the event log is to read the entire event log
so I'd say, on those meters, maybe we should read the events once a week?
there are more than 10k ACE6000s....

ok, so for now, I'm just going to switch off events on all the meters

>We should read profiles,  totals and billing registers for all meters

even events and possibly billing registers, maybe even totals. I think we should focus on profile only for now
because maxdate is profile
so if we want to get the number of meters read to be more quicker, we can do profile only. That is the most important.
of course, without totals, the readings would be wrong, so that is also important.

>Ok

and then once we have profile and totals working, we switch on billing registers

> Ok so profiles and totals for now

well, if we have like, the week end, I'd say, lets focus on only profile. To get back in control.
then later today or early 2moro, we switch on totals
then once totals look good, we switch on billing registers also

ok, bulk edit is going through...
the amount of concurrent queries are a lot more now than yesterday. All the Output threads are busy.

do we have to have the TOU registers in the recon report?
wouldn't it be better to first reconcile the energy totals, before we start getting into the TOU registers?
because the TOU registers are non-standard, meaning there could be config errors or differences on meters. But the default energy registers and global kVA are stock standard, and more of a hard limit te measure the accuracy of the system.

> I needed it in a comprehensive report in order to submit both TOU and NOU meters in that report

all I am saying, is that if you want to prove data integrity, you start from the foundation up, and you don't build the walls before you lay the foundation

there is a progression to this

> ok, I hear you

However, I am a developer, and I don't want my baby to have trauma. Also, since the customer is looking at us, we have a limited tolerance for failure
we are trying to prove data integrity

Also, if we start with a system that is not working, it is difficult to know what is wrong. It is easier to start with a simpler setup, that works, and as we incrementally add things, if it doesn't work or performs badly, we know where to concentrate. It is difficult to troubleshoot a large setup that is not working.

If you want to have a faster rollout, your method does have merit: it is like SpaceX blowing up Starships and learning faster. So we could also explain it to the customer in these terms. Since they wanted to go live quick

ok, I've run the profile only setting for the meters

we're on list sz:6627(6627);csd sz:0;outpbsz:1345)
that means, 6000+ meters still to call in, and 1345 meters in the output buffer

Let me restart the server, and then we see the performance...

(if a meter time is a little behind, with the 5 minutes deadband you make sure you read the last bit of e.g. the last day when you read, and don't leave out the last half hour profile period when you read)

With billing in South Africa typically working from half hour profile anyway, it doesn't really benefit you to set the Acquire Times more often. I mean, just the effort of running through tens of thousands of meters and trying to detect accross all sets which ones are not 'up to date', takes a significant amount of processing. If you basically do that constantly all the time the system won't have time to do anything else.

For people who want real time values, that is supported but by a completely different sub-system. That is typically used in factories where the server and meters are in the same building, over a LAN, and efficient meter energy value updates happen through SNMP. If you have breakers that needs to be switched with higher responsiveness, you can put an entry every other 5 minutes to schedule Undone Tasks. But pls don't schedule actual meter readings every 5 minutes pls, that makes the system perform badly.