= Change Requested = Applied the following patch to /usr/lib/python2.6/site-packages/fas/templates/user/view.html on fas0[1-3] to work around 500s on /user/view:
{{{ --- /usr/lib/python2.6/site-packages/fas/templates/user/view.html 2010-08-27 15:57:26.000000000 +0000 +++ /home/fedora/ricky/view.html 2010-08-31 05:26:14.136666045 +0000 @@ -123,8 +123,8 @@
This patch doesn't actually address the root of the problem, so we will need to investigate further. The full traceback for the 500s was:
{{{ cherrypy.msg INFO HTTP: Page handler: <bound method User.view of <fas.user.User object at 0xb4203c2c>> Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/CherryPy-2.3.0-py2.6.egg/cherrypy/cphttptools.py", line 121, in _run self.main() File "/usr/lib/python2.6/site-packages/CherryPy-2.3.0-py2.6.egg/cherrypy/_cphttptools.py", line 264, in main body = page_handler(virtual_path, self.params) File "<string>", line 3, in view File "/usr/lib/python2.6/site-packages/turbogears/identity/conditions.py", line 210, in require return fn(self, *args, kwargs) File "<string>", line 3, in view File "/usr/lib/python2.6/site-packages/turbogears/controllers.py", line 195, in validate return errorhandling.run_with_errors(errors, func, args, kw) File "/usr/lib/python2.6/site-packages/turbogears/errorhandling.py", line 119, in run_with_errors return func(self, *args, kw) File "<string>", line 3, in view File "/usr/lib/python2.6/site-packages/turbogears/controllers.py", line 360, in expose *args, kw) File "<string>", line 5, in run_with_transaction File "/usr/lib/python2.6/site-packages/turbogears/database.py", line 411, in sa_rwt retval = func(*args, kw) File "<string>", line 5, in _expose File "/usr/lib/python2.6/site-packages/turbogears/controllers.py", line 373, in <lambda> mapping, fragment, args, kw))) File "/usr/lib/python2.6/site-packages/turbogears/controllers.py", line 420, in _execute_func return _process_output(output, template, format, content_type, mapping, fragment) File "/usr/lib/python2.6/site-packages/turbogears/controllers.py", line 88, in _process_output fragment=fragment) File "/usr/lib/python2.6/site-packages/turbogears/view/base.py", line 159, in render return engine.render(kw) File "/usr/lib/python2.6/site-packages/genshi/template/plugin.py", line 110, in render return self.transform(info, template).render(kwargs) File "/usr/lib/python2.6/site-packages/genshi/core.py", line 179, in render return encode(generator, method=method, encoding=encoding, out=out) File "/usr/lib/python2.6/site-packages/genshi/output.py", line 60, in encode return _encode(u''.join(list(iterator))) File "/usr/lib/python2.6/site-packages/genshi/output.py", line 311, in call for kind, data, pos in stream: File "/usr/lib/python2.6/site-packages/genshi/output.py", line 592, in call for kind, data, pos in stream: File "/usr/lib/python2.6/site-packages/genshi/output.py", line 698, in call for kind, data, pos in chain(stream, [(None, None, None)]): File "/usr/lib/python2.6/site-packages/genshi/output.py", line 532, in call for ev in stream: File "/usr/lib/python2.6/site-packages/genshi/core.py", line 283, in _ensure for event in stream: File "/usr/lib/python2.6/site-packages/genshi/template/base.py", line 569, in _include for event in stream: File "/usr/lib/python2.6/site-packages/genshi/template/markup.py", line 279, in _match content = list(content) File "/usr/lib/python2.6/site-packages/genshi/template/base.py", line 569, in _include for event in stream: File "/usr/lib/python2.6/site-packages/genshi/template/markup.py", line 245, in _match for event in stream: File "/usr/lib/python2.6/site-packages/genshi/template/markup.py", line 234, in _strip event = stream.next() File "/usr/lib/python2.6/site-packages/genshi/template/base.py", line 543, in _exec for event in stream: File "/usr/lib/python2.6/site-packages/genshi/template/base.py", line 496, in _eval for kind, data, pos in stream: File "/usr/lib/python2.6/site-packages/genshi/template/base.py", line 558, in _flatten for event in self._flatten(substream, ctxt, vars): File "/usr/lib/python2.6/site-packages/genshi/template/base.py", line 558, in _flatten for event in self._flatten(substream, ctxt, vars): File "/usr/lib/python2.6/site-packages/genshi/template/base.py", line 557, in _flatten vars) File "/usr/lib/python2.6/site-packages/genshi/template/base.py", line 272, in _apply_directives stream = directives0 File "/usr/lib/python2.6/site-packages/genshi/template/directives.py", line 411, in call value = _eval_expr(self.expr, ctxt, vars) File "/usr/lib/python2.6/site-packages/genshi/template/base.py", line 286, in _eval_expr retval = expr.evaluate(ctxt) File "/usr/lib/python2.6/site-packages/genshi/template/eval.py", line 180, in evaluate return eval(self.code, _globals, {'data': data}) File "/usr/lib/python2.6/site-packages/fas/templates/user/view.html", line 126, in <Expression u'role.group in person.approved_memberships'> ${('Approved')} File "/usr/lib/python2.6/site-packages/sqlalchemy/ext/associationproxy.py", line 356, in contains for member in self.col: File "/usr/lib/python2.6/site-packages/sqlalchemy/ext/associationproxy.py", line 300, in <lambda> col = property(lambda self: self.lazy_collection()) File "/usr/lib/python2.6/site-packages/sqlalchemy/ext/associationproxy.py", line 158, in lazy_collection "stale association proxy, parent object has gone out of " InvalidRequestError: stale association proxy, parent object has gone out of scope }}}
http://www.mail-archive.com/sqlalchemy@googlegroups.com/msg08914.html
Ran the test from that thread and I don't see the problem so this is different in some way.
This also affects the json return values which is problematic for fedoracommunity and probably python-fedora. lmacken, any ideas on what this could be?
{{{ [Tue Aug 31 22:54:01 2010] [error] Traceback (most recent call last): [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/CherryPy-2.3.0-py2.6.egg/cherrypy/_cphtt ptools.py", line 121, in _run [Tue Aug 31 22:54:01 2010] [error] self.main() [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/CherryPy-2.3.0-py2.6.egg/cherrypy/_cphtt ptools.py", line 264, in main [Tue Aug 31 22:54:01 2010] [error] body = page_handler(virtual_path, self.params) [Tue Aug 31 22:54:01 2010] [error] File "<string>", line 3, in list [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/turbogears/identity/conditions.py", line 210, in require [Tue Aug 31 22:54:01 2010] [error] return fn(self, *args, kwargs) [Tue Aug 31 22:54:01 2010] [error] File "<string>", line 3, in list [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/turbogears/controllers.py", line 360, in expose [Tue Aug 31 22:54:01 2010] [error] args, kw) [Tue Aug 31 22:54:01 2010] [error] File "<string>", line 5, in run_with_transaction [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/turbogears/database.py", line 411, in sa_rwt [Tue Aug 31 22:54:01 2010] [error] retval = func(*args, kw) [Tue Aug 31 22:54:01 2010] [error] File "<string>", line 5, in _expose [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/turbogears/controllers.py", line 252, in <lambda> [Tue Aug 31 22:54:01 2010] [error] None, False, args, kw)) [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/turbogears/controllers.py", line 420, in _execute_func [Tue Aug 31 22:54:01 2010] [error] return _process_output(output, template, format, content_type, mapping, fragment) [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/turbogears/controllers.py", line 88, in _process_output [Tue Aug 31 22:54:01 2010] [error] fragment=fragment) [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/turbogears/view/base.py", line 159, in render [Tue Aug 31 22:54:01 2010] [error] return engine.render(kw) [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/turbojson/jsonsupport.py", line 40, in render [Tue Aug 31 22:54:01 2010] [error] output = self.encoder.encode(info) [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/simplejson/encoder.py", line 200, in encode [Tue Aug 31 22:54:01 2010] [error] chunks = self.iterencode(o, _one_shot=True) [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/simplejson/encoder.py", line 260, in iterencode [Tue Aug 31 22:54:01 2010] [error] return _iterencode(o, 0) [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/turbojson/jsonify.py", line 163, in default [Tue Aug 31 22:54:01 2010] [error] return jsonify(obj) [Tue Aug 31 22:54:01 2010] [error] File "<generated code>", line 0, in jsonify [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/peak/rules/core.py", line 153, in call [Tue Aug 31 22:54:01 2010] [error] return self.body(*args, kw) [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/fedora/tg/json.py", line 156, in jsonify_salist [Tue Aug 31 22:54:01 2010] [error] return [jsonify(element) for element in obj] [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/sqlalchemy/ext/associationproxy.py", line 380, in iter [Tue Aug 31 22:54:01 2010] [error] for member in self.col: [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/sqlalchemy/ext/associationproxy.py", line 300, in <lambda> [Tue Aug 31 22:54:01 2010] [error] col = property(lambda self: self.lazy_collection()) [Tue Aug 31 22:54:01 2010] [error] File "/usr/lib/python2.6/site-packages/sqlalchemy/ext/associationproxy.py", line 158, in lazy_collection [Tue Aug 31 22:54:01 2010] [error] "stale association proxy, parent object has gone out of " [Tue Aug 31 22:54:01 2010] [error] InvalidRequestError: stale association proxy, parent object has gone out of scope [Tue Aug 31 22:54:01 2010] [error] [Tue Aug 31 22:54:01 2010] [error] turbogears.access INFO - toshio "GET /accounts/user/list/U*?tg_format=json&_csrf_token=3b593877564f86ebda28d23d46a1a3c0a70e3d22 HTTP/1.0" 500 791 "" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.7) Gecko/20100723 Fedora/3.6.7-1.fc13 Firefox/3.6.7" }}}
This is replicable in the staging environment. For instance:
https://admin.stg.fedoraproject.org/accounts/user/list/U*?tg_format=json
I also tested the latest 0.5.x version of sqlalchemy but the issue does not go away: python-sqlalchemy-0.5.8-3.el6.noarch.rpm
(Built this in koji and tested it on fas01.stg. After testing, reverted to 0.5.5)
Okay, attaching a new hotfix patch -- this was applied on top of the other patch but this is the only one that should be sufficient if we have to rebuild a server. lmacken, please test fedoracommunity, this should fix json and html for all URLs.
Run list() on all data froman association_proxy on person object after filter_private has been run. 0001-Fix-at-the-model-controller-level-for-association_pr.patch
New fix because the last fix was too slow when querying for all of the people who have signed cla_done. I've converted the method to use the SQL level API instead of the ORM. I had to replicate a bit of filter_private code to do so. We might want to try abstracting that at some point in the future.
new /usr/lib/python2.6/site-packages/fas/user.py user.py
This change is in the latest fas release which is now deployed.
Login to comment on this ticket.