Fortunately we won’t have to worry about splitting the files as I am only interested in maintaining a list of msg types and an incrementing number of occurances. I won’t actually be sending the data of all those files. So I should be ok with reading a line at a time into the same variable. identify the data i need. Store the precise data in a list with its counter.
At any given time I should not have more than a single message plus a list of msg types and counters in memory.
That is my thought anyways…
As you process your files, can you do an append to the existing file vs combining the files in memory…. I think you might be able to conserve some there… Just guessing.
Welcome to other thoughts and suggestions 😆
Ryan
