#1225 [RFE] Need file upload mechanism
Opened 12 years ago by edewata. Modified 7 years ago

The Web UI needs a server-side mechanism to upload a file (e.g. certificate) into the server and attach the file to an entry (e.g. host, service, entitlement) in the server. The CLI doesn't have this problem because it can access the content of the file directly whereas the browser cannot due to security restrictions.

So an IPA operation that takes a file will be done in 2 steps:
- The browser uploads the file to the server. The server stores the file in a temporary storage. The server returns a file ID back to the browser.
- The browser calls the operation with the file ID. The server reads the content of the file and executes the operation with it. The file is then removed from the temporary storage.

The mechanism needs to address these issues:
- Access control: Which users/groups can upload?
- Privacy: Each user should have a private temporary storage on the server.
- Conflicts: Can the user upload the same file into 2 browser instances to execute different operations?
- Management: What to do with uploaded files that are not attached to anything after sometime? Can user browse uploaded files?
- Limits: Is there a limit on the size, type, or number of files?
- Replication: Will the temporary storage be replicated? In a replicated system the file could be uploaded to one server while the operation is executed on another.
- API change: How to specify the file ID into the IPA operations?


Access control. This should be along the same rules as the self-service vs full navigation: you need to have some role. For now, I'd say admin, and then add in an additional "file upload" privilege and permission to roles below that.

Privacy: not yet necessary, but lets plan for it. Since the data is going to end up in the DirSrv eventually, and it is globally readable, we should be OK letting one user see anthers temporary state. However, if we end up doing anything with private keys or password type mechanisms, then it will be necessary.

Conflicts: for now, last one wins.

Management: files should not stay around that long. Ideally, this would be tied to session timeout, but I'd say that a file should only stay on the server until it is read, after which it is quickly deleted.

Limits: text only, with a configurable max size. The default max size should be able to hold an X509 Certificate plus a small amount of Data.

Replication: No. Files are part of the RPC, but not the end state. a file is only uploaded to then be transferred into an attribute in the dirsrv.

API: Store the file in a dictionary keyed on userid:entity:pkey:field. To upload, do an HTTP post, to get the file back, do an HTTP get

The mechanism needs to address these issues:
* Access control: Which users/groups can upload?

Anyone authenticated can upload.
The actual operation against the uploaded file should define the access control.
This can be done by creating a special "upload config" entry in the DS and reading effective rights of this entry. The rights defined against the entry will be applied to the file operation. We already have similar approach regarding certificate issuance.

  • Privacy: Each user should have a private temporary storage on the server.

It is a temp storage and should be destroyed as part of the reboot. The server side operation should move the file to the right place or insert into an LDAP entry. It depends on the operation. I do not see a need for the individual storage as long as the names of the temp files are unique.

  • Conflicts: Can the user upload the same file into 2 browser instances to execute different operations?

Yes. They will have different names. The file can be uploaded and then abandoned. The reboot will clear the temp on the server side.

  • Management: What to do with uploaded files that are not attached to anything after sometime? Can user browse uploaded files?

Destroy. No browsing. Files are uploaded to be processed in some way.

  • Limits: Is there a limit on the size, type, or number of files?

Yes. Hardcoded on the server. Would be nice to have some type checking heuristics.

  • Replication: Will the temporary storage be replicated? In a replicated system the file could be uploaded to one server while the operation is executed on another.

No replication. The actual server side operation should move the file into the area that is replicated, shared or a part of DS.

  • API change: How to specify the file ID into the IPA operations?

mktemp output is the ID of the uploaded file.

Replying to [comment:2 dpal]:

The mechanism needs to address these issues:
* Access control: Which users/groups can upload?

Anyone authenticated can upload.
The actual operation against the uploaded file should define the access control.
This can be done by creating a special "upload config" entry in the DS and reading effective rights of this entry. The rights defined against the entry will be applied to the file operation. We already have similar approach regarding certificate issuance.

ACK assuming it is easy to do.

  • Privacy: Each user should have a private temporary storage on the server.

It is a temp storage and should be destroyed as part of the reboot. The server side operation should move the file to the right place or insert into an LDAP entry. It depends on the operation. I do not see a need for the individual storage as long as the names of the temp files are unique.

We do not have any storage that is cleanable on reboot at this time, we do not even have a safe way to detect we have a reboot. Plus a reboot may not happen for months.
A time base expiration and a cleanup process (even a simple cron job) are probably safer.

  • Conflicts: Can the user upload the same file into 2 browser instances to execute different operations?

Yes. They will have different names. The file can be uploaded and then abandoned. The reboot will clear the temp on the server side.

This can easily cause high disk usage, we should do time based clean-up, and unless there is a very good reason to keep multiple files around I agree with Adam that only one file should be kept around.

  • Management: What to do with uploaded files that are not attached to anything after sometime? Can user browse uploaded files?

Destroy. No browsing. Files are uploaded to be processed in some way.

Agreed.

  • Limits: Is there a limit on the size, type, or number of files?

Yes. Hardcoded on the server. Would be nice to have some type checking heuristics.

Perhaps have a callback to the code that will later process the file to get the expected maximum size ?

  • Replication: Will the temporary storage be replicated? In a replicated system the file could be uploaded to one server while the operation is executed on another.

No replication. The actual server side operation should move the file into the area that is replicated, shared or a part of DS.

ack

  • API change: How to specify the file ID into the IPA operations?

mktemp output is the ID of the uploaded file.

The question was about how we represent the ID.

If we store data in a private directory we do not need to use mktemp and we can use a predictable name keyed to the user/operation name. This way we do not need any ID at the API level as the name can be easily reconstructed.

This is quite old ticket resurrected from Deferred - resetting an owner.

Metadata Update from @edewata:
- Issue assigned to someone
- Issue set to the milestone: Future Releases

7 years ago

Login to comment on this ticket.

Metadata