Time for an internal reshuffle. My initial design wasn’t working.
Initially I figured that the front page search could have its own index, which stores individual comics (each day’s strip, perhaps) as a single clump, not worrying about who said it or anything.
But since I’ve got the advanced search up and running in the last couple of days I’ve realised that it isn’t working very well, because it uses a separate index to handle the text broken down by speaker.
Not to mention that the parser eats memory, but I’ll talk more about that in a minute.
So now there’s only the one index, which speeds up indexing.
But what about eating memory? Well, I’m sure the parser script is running how it is supposed to - it’s doing everything it should, but somehow it’s eating memory at an alarming rate, and fails at about 100 strips, after consuming 128MB of RAM. I’ve never seen a script eat this much.
I’ll figure it out though, but the work gone into indexing up 100 strips over the last couple of weeks is going to have to slow. I need to figure the code out first.
Current Mood:
disappointed