= phenomenon =
mirrorlist_server was consuming 500MB RAM, leading to much slower fork() times, thus slow response times, and problems with concurrent connections. Could also be why app7 had to be taken out of the rotatation, as it has less RAM than the other app servers.

= reason =
python file.readlines() used instead of readline() to read the global_netblocks file. readlines() consumes 500MB RAM, instead of 50 for readline().

= recommendation =

From dfac59d0608481df0c8d759551509a8c4cee23f3 Mon Sep 17 00:00:00 2001
From: Matt Domsch matt@domsch.com
Date: Wed, 30 Sep 2009 16:08:01 -0500
Subject: [PATCH] mirrorlist_server: use file.readline() instead of readlines()

readlines() takes 500MB RSS to read in an 8MB 375k-line text file.
readline() takes 100MB. Go figure.

mirrorlist-server/mirrorlist_server.py | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/mirrorlist-server/mirrorlist_server.py b/mirrorlist-server/mirrorlist_server.py
index 1dae32d..4314dc3 100755
--- a/mirrorlist-server/mirrorlist_server.py
+++ b/mirrorlist-server/mirrorlist_server.py
@@ -646,7 +646,7 @@ def setup_netblocks(netblocks_file):
if netblocks_file is not None:
try:
f = open(netblocks_file, 'r')
- for l in f.readlines():
+ for l in f.readline():
s = l.split()
start, mask = s[0].split('/')
mask = int(mask)
--
1.6.0.6