Bug #4958
closed
java.io.IOException: Pipe closed when registering file with some special characters to LFC
0%
Description
Whenever I try to store a file with characters like " " (space), "%" or "{" (I guess not only these) on LFC, I get the error "java.io.IOException: Pipe closed". The problem, however, does not appear with characters like "'", "(", "+", "=", "&" and some others.
What is even more disturbing, the files for which the IOException was thrown get uploaded to SRM - with space replaced by "%20", "%" by "%25" and so on (other characters I mentioned as not causing the problem are not escaped). Therefore, when I retry to upload them I get an error that they already exist.
The behaviour of the registration is inconsistent and I found no information as to which characters are invalid for the LFC layer. Also the reported error gives no information as to the actual cause of the problem.
When using the standard LFC command-line tools, all the characters I mentioned are accepted without any problem.
Here is the code I use for registering the file:
//copying local file to remote SE
localEntry = NSFactory.createNSEntry(session, source, Flags.NONE.getValue());
localEntry.copy(srmTarget, Flags.NONE.getValue());
//creating and registering a new LFC entry
lfcFile = LogicalFileFactory.createLogicalFile(session, lfcTarget, Flags.CREATE.or(Flags.EXCL));
lfcFile.addLocation(srmTarget);
source, srmTarget, lfcTarget were created with URLFactory.createURL(fullURL);
I attach the full stack trace (pl.cyfronet.insilicolab.server.scripting.integration.lfc.Lfc.storeFile(Lfc.java:154) points to line: localEntry.copy(srmTarget, Flags.NONE.getValue());).
Files
Updated by Schwarz Lionel over 11 years ago
- Category set to gLite adaptors
- Assigned To set to Schwarz Lionel
Thanks Joanna for reporting.
Could you provide the target version you had the issue with 0.9.16 or 0.9.17
I'll have a look today
Cheers
Lionel
Updated by Kocot Joanna over 11 years ago
Hi Lionel,
I am using 0.9.15 as this was the last non-SNAPSHOT version and I need a stable version for my project.
Cheers,
Joanna
Updated by Schwarz Lionel over 11 years ago
0.9.15 is too old. Even if there was no official release for 0.9.16 you can safely use it. It has many bug fixes.
If, for technical reasons, you need a release (no SNAPSHOT) for 0.9.16, let us know.
Lionel
Updated by Kocot Joanna over 11 years ago
Ok, I will check with the 0.9.16 version and let you know if the problem still occurs.
Joanna
Updated by Kocot Joanna over 11 years ago
With 0.9.16-SNAPSHOT the problem is still there.
Joanna.
Updated by Schwarz Lionel over 11 years ago
Joanna could you please send me (lionel.schwarz AT in2p3.fr) the URLs you use (LFC and SRM)
Thanks
Lionel
Updated by Kocot Joanna over 11 years ago
I used srm://dpm.cyf-kr.edu.pl/dpm/cyf-kr.edu.pl/home/vo.cta.in2p3.fr/insilicolab-test/c.PL__o.GRID__o.Cyfronet__cn.Joanna_Kocot_-_PL-Grid/ and /grid/vo.cta.in2p3.fr/insilicolab-test/c.PL__o.GRID__o.Cyfronet__cn.Joanna_Kocot_-_PL-Grid/ as LFC (server: lfc-e). And I tried to send e.g. "test with spaces.txt" or "test%.txt". So the "srmTarget" from my code is e.g. srm://dpm.cyf-kr.edu.pl/dpm/cyf-kr.edu.pl/home/vo.cta.in2p3.fr/insilicolab-test/c.PL__o.GRID__o.Cyfronet__cn.Joanna_Kocot_-_PL-Grid/test%20with%20spaces.txt. The file gets created on the SRM but is empty, so the error is somwhere in the copy() method.
I guess the trouble is with character '%', and all the characters that get encoded when creating the URL - because they are encoded to %<something>.
Joanna
Updated by Schwarz Lionel over 11 years ago
- Status changed from New to Feedback
Joanna,
There is a new snapshot with fixes for SRM and LFC adaptors.
However I could not reproduce your issue with " ", "%" and "{" characters in the logical filename.
Lionel
Updated by Kocot Joanna over 11 years ago
Hi Lionel,
To summarize: the error happens every time I do a NSEntry.copy() to an SRM URL that contains '%' character (the NSEntry is also constructed with a URL containing '%').
I will also do some tests with the new snapshot.
Thanks,
Joanna
Updated by Schwarz Lionel over 11 years ago
- Status changed from Feedback to Resolved
Updated by Kocot Joanna about 11 years ago
In the new JSAGA (version 1.0) the exception "Pipe closed" is not thrown anymore, however, there is still a problem with files containing '%' and other characters that get encoded to '%XX'. This ends up with a DoesNotExist exception, but I think this is not a JSAGA issue anymore, only SRM not dealing correctly with '%' characters.
Updated by Kocot Joanna about 11 years ago
Thinking of this problem again, I think that the URLs that are targeted for the SRM should not be encoded as they are now. The class fr/in2p3/jsaga/helpers/URLEncoder.java is used here, and replaces chars: '"', '%', '<', '>', '[', '\\', ']', '^', '`', '{', '|', '}', 127 with '%<HexValue>'. As the SRM does not accept '%' sign, this makes not only the '%', but all the mentioned special signs not being accepted for LFC registration.
Maybe you should consider using different encoding for these URLs? Shall I start a new ticket for that?
Updated by Schwarz Lionel about 11 years ago
Hi Joanna,
You might consider using latest versions of JSAGA. The release 1.0.1 (which will only be fixed for important issues):
http://software.in2p3.fr/jsaga/latest-release/download.html
Or preferably the current snapshot 1.1.0 where I could fix your issue very quickly:
http://software.in2p3.fr/jsaga/dev/download.html
If the '%' issue is still there, please open a new bug with the full description. I'll have a look asap.
Cheers, Lionel