Steps to reproduce:
while true; do; echo ipa <someaction> & [ $(jobs |wc -l) -ge 64 ] && wait ; done
... and wait.
Eventually, ~/.cache/ipa/servers/<servername> will become corrupt, and all subsequent IPA commands will exit with the following error:
ipa: ERROR: ValueError: Extra data: line 1 column 75 - line 2 column 1 (char 74 - 76) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ipalib/cli.py", line 1348, in run api.finalize() File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 707, in finalize self.__do_if_not_done('load_plugins') File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 422, in __do_if_not_done getattr(self, name)() File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 585, in load_plugins for package in self.packages: File "/usr/lib/python2.7/site-packages/ipalib/__init__.py", line 919, in packages ipaclient.remote_plugins.get_package(self), File "/usr/lib/python2.7/site-packages/ipaclient/remote_plugins/__init__.py", line 112, in get_package server_info = ServerInfo(api) File "/usr/lib/python2.7/site-packages/ipaclient/remote_plugins/__init__.py", line 39, in __init__ self._read() File "/usr/lib/python2.7/site-packages/ipaclient/remote_plugins/__init__.py", line 44, in _read self._dict = json.load(sc) File "/usr/lib64/python2.7/json/__init__.py", line 290, in load **kw) File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 369, in decode raise ValueError(errmsg("Extra data", s, end, len(s))) ValueError: Extra data: line 1 column 75 - line 2 column 1 (char 74 - 76) ipa: ERROR: an internal error has occurred
Deleting .cache will remedy the situation (until you run enough parallel transactions for it to happen again).
Another workaround is to chown the JSON file to something the user cannot write to. This has the side effect of every IPA command WARNING about not being able to edit it. While this does appear to work, I do not have the faintest clue as to any possible implications.
Just for the record, file ~/.cache/ipa/servers/<servername> contains information about expiration time stamp and schema fingerprint (and also schema language but that's not important right now).
When the expiration time stamp is in past CLI calls 'schema' command to verify that the stored fingerprint is still up to date (and download newer when available) and then updates the expiration time stamp in ~/.cache/ipa/servers/<servername>.
If it's unsuccessful in updating the file (because you changed privileges or ownership) the expiration time stamp will remain in past and every consecutive call will be delayed with the check again. But everything will work, just slower.
master:
Metadata Update from @umlorime: - Issue assigned to dkupka - Issue set to the milestone: FreeIPA 4.5
The fix simply ignores the corruption. But there's a significant performance penalty once it happens, which seems like it will continue forever. I'd rather see it delete the corrupted cache. Even better, use an approach that won't result in corruption. Linux certainly has good enough locking to support that.
This is an issue for us in production. We're fixing it by setting XDG_CACHE_HOME to a unique value. But I'd rather see a real solution.
Login to comment on this ticket.