I’ve some problem with the Golang GC (even if I shouldn’t). I’ve a system who read from the database a chunk of data, it split in a smaller chunk, and it analyses it using the go-routine. When it has done it read the next chunk of data from the database, and it repeats the process. This system has a limited amount of memory, and I can’t expand it. What I see is the system start with an X amount of memory used, and then from time to time it spikes up, but it doesn’t release anything. The main routine is sync with the go-routine, so it does not start to manage a new chunk until the older is done. Each goroutine copies the smaller chunk of data so soon as all the smaller chunk are delivered to the go-routine the main routine can retrieve the new chunk. I reuse the go-routine, so I’ve a fixed number of goroutine at any given time allocated all once and then orchestrated.
I really don’t get it what I’m doing wrong. Someone have any idea about what I can do?