\r\n/// <summary>\r\n/// Adds the SPFile to the zip output stream.\r\n/// </summary>\r\n/// <param name="zipOutputStream">The zip output stream.</param>\r\n/// <param name="file">The file.</param>\r\n/// <param name="path">The path.</param>\r\npublic static void AddSPFile(this ZipOutputStream zipOutputStream, SPFile file, string path) {\r\n string filePath = Path.Combine(path, file.Name);\r\n\r\n ZipEntry entry = new ZipEntry(filePath);\r\n entry.DateTime = file.TimeCreated;\r\n zipOutputStream.PutNextEntry(entry);\r\n\r\n using (Stream fileStream = file.OpenBinaryStream()) {\r\n byte[] buffer = new byte[4096];\r\n int sourceBytes;\r\n do {\r\n sourceBytes = fileStream.Read(buffer, 0, buffer.Length);\r\n zipOutputStream.Write(buffer, 0, sourceBytes);\r\n } while (sourceBytes > 0);\r\n }\r\n}\r\n\r\n\r\n",
"PostedDate": "2009-03-01T03:35:50.89-08:00",
"UserRole": null,
"MarkedAsAnswerDate": null
},
{
"Id": "163193",
"ThreadId": "48865",
"Html": "Webbes,
\r\nI understand your situation. I have to think about it a little. Just at the outset, I have a couple ideas.
\r\nYou CAN stream the contents of the zip entry immediately, as soon as you add the entry. To do this, you would call Save() immediately after adding each file or stream. This would cause the current stream being added to be read from the database, and then written to the zip archive on disk. On subsequent calls to Save(), the same thing happens with the current entry, but the data for the previously-written entries is just transferred from one zip file to another. This seems like it would work, avoiding the 100 simultaneous connections to the database, though of course, each file saved to the zip gets written and rewritten, (n-i) times. The first one, 100 times. The second entry 99 times, etc. Seems very inefficient. But it would avoid the simultaneous db connections problem.
\r\n
\r\nA real solution to your problem would eliminate the simultaneous db connections while at the same time not produce any redundant data writes.
\r\nThere are a couple ways I could address this.
There may be other ways, too.
\r\n
\r\nWhat do you think of the first option?
The solution with events would work and I see you've created a workitem for it. Maybe I'll see if i can do something with it myself.
\r\nThe first option is not really an option. I can't create 100 zipfiles before I send one. It would be to resource intensive. My solution works in a SharePoint enterprise portal and must be very well performing.
\r\nCheers,
\r\nWes
\r\n", "PostedDate": "2009-03-01T08:46:54.19-08:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "163218", "ThreadId": "48865", "Html": "ok, makes sense. Well, have a look at the event code.I think there would be some loose ends to clean up, but that would be the basic idea.
\r\n", "PostedDate": "2009-03-01T09:48:29.647-08:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "163742", "ThreadId": "48865", "Html": "\r\n\r\n\r\n
I tried the reverse: delay opening the file like in zip.AddFileStream(fileName, string.Empty, new LazyStream(() => catalog.Open(fileName)));
\r\n\r\n
\r\n
\r\nI expected the Save method to close the file, but it is not the case and now I can't close it because I have no Stream property on the ZipEntry
\r\n
\r\nwith the following LazyStream class (the delegate is the same as Func<Stream> but can be used with .net 2.0)
\r\n
\r\n
\r\npublic
delegate Stream StreamOpener();\r\n
public class LazyStream : Stream\r\n
{
\r\n\r\n\r\n
private StreamOpener Opener;\r\n
private Stream BaseStream;\r\n
public LazyStream(StreamOpener opener)\r\n {
if (opener == null)\r\n
throw new ArgumentNullException();\r\n Opener = opener; }
public override bool CanRead\r\n {
get { return true; }\r\n }
public override bool CanSeek\r\n {
get { return true; }\r\n }
public override bool CanWrite\r\n {
get { return false; }\r\n }
public override void Flush()\r\n {
throw new NotImplementedException();\r\n }
public override long Length\r\n {
get\r\n
{
\r\n\r\n\r\n
if (BaseStream == null)\r\n BaseStream = Opener();
return BaseStream.Length;\r\n } }
public override long Position\r\n {
get\r\n
{
\r\n\r\n\r\n
if (BaseStream == null)\r\n
return 0;\r\n
return BaseStream.Position;\r\n }
set\r\n
{
\r\n\r\n\r\n
if (value != 0)\r\n
throw new ArgumentOutOfRangeException();\r\n
if (BaseStream != null && BaseStream.Position != 0)\r\n { BaseStream.Close(); BaseStream =
null;\r\n } } }
public override int Read(byte[] buffer, int offset, int count)\r\n {
if (BaseStream == null)\r\n BaseStream = Opener();
if (!BaseStream.CanRead)\r\n
throw new InvalidOperationException();\r\n
int n = 0;\r\n
int m;\r\n
while (count != 0 && (m = BaseStream.Read(buffer, offset, count)) != 0)\r\n { offset += m; n += m; count -= m; }
return n;\r\n }
public override long Seek(long offset, SeekOrigin origin)\r\n {
if (origin == SeekOrigin.Current)\r\n {
if (offset != 0)\r\n
throw new ArgumentOutOfRangeException();\r\n
if (BaseStream == null)\r\n
return 0;\r\n
return BaseStream.Position;\r\n }
if (origin != SeekOrigin.Begin)\r\n
throw new ArgumentOutOfRangeException();\r\n
if (offset != 0)\r\n
throw new ArgumentOutOfRangeException();\r\n
if (BaseStream != null)\r\n BaseStream.Close(); BaseStream =
null;\r\n
return 0;\r\n }
public override void SetLength(long value)\r\n {
throw new NotImplementedException();\r\n }
public override void Write(byte[] buffer, int offset, int count)\r\n {
throw new NotImplementedException();\r\n }
public override void Close()\r\n {
if (BaseStream != null)\r\n BaseStream.Close(); } }
Hmm, interesting.
\r\nBut yes, DotNetZip does not close streams that are presented to it from external sources.
I have to think about this some more.
\r\n", "PostedDate": "2009-03-03T06:44:20.4-08:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "164490", "ThreadId": "48865", "Html": "Hey Webbes,
\r\nI thought about this some more and came up with this idea. It introduces a new interface into DotNetZip - IStreamDispenser - which exposes just two methods: Open and Return. DotNetZip calls the Open method when it needs a stream and calls the Return when it is done.
\r\nYou use it this way:
void IStreamDispenser.Return(System.IO.Stream s, string name)\r\n{\r\n Console.WriteLine("closing {0}", name);\r\n s.Close();\r\n}\r\n\r\n\r\nSystem.IO.Stream IStreamDispenser.Open(string name)\r\n{\r\n Console.WriteLine("opening {0}", name);\r\n System.IO.Stream s = System.IO.File.OpenRead(System.IO.Path.Combine("fodder",name));\r\n return s;\r\n}\r\n\r\n\r\npublic void Run()\r\n{\r\n using (ZipFile zip = new ZipFile())\r\n {\r\n\tforeach (var f in System.IO.Directory.GetFiles(DirectoryToZip))\r\n\t zip.AddFileStream(f, "", null);\r\n\tzip.StreamDispenser = this;\r\n\tzip.Save(ZipArchiveName);\r\n }\r\n}\r\n\r\n
The StreamDispenser methods are called once for each entry, and only within the scope of the ZipFile.Save(), and only when the stream you have passed origially is null.
\r\n
\r\nCouple questions:
If you want to try out the StreamDispenser, grab v1.8.1.9 .
\r\nAnd I'm interested in your comments on the design.
This could definately work for me, and I'm happy with this option already.
\r\nI would prefer events though. I would create something like this (if I had the time to really get into the library. I don't know if I can really access the underlying Stream of a ZipEntry for example) :
\r\nAdditions to the zip class:
\r\n\r\npublic event EventHandler<StreamSavingEventArgs> SavingStream;\r\npublic event EventHandler<StreamSavingEventArgs> SavedStream;\r\n\r\nprotected virtual void OnSavingStream(StreamSavingEventArgs eventArgs){\r\n EventHandler<StreamSavingEventArgs> temp = SavingStream;\r\n if(temp != null){\r\n temp(this, eventArgs);\r\n }\r\n}\r\n\r\nprotected virtual void OnSavedStream(StreamSavingEventArgs eventArgs){\r\n EventHandler<StreamSavingEventArgs> temp = SavedStream;\r\n if(temp != null){\r\n temp(this, eventArgs);\r\n }\r\n}\r\n\r\n//I imagine there's something like a SaveStream method\r\nprivate void SaveStream(ZipEntry zipEntry){\r\n StreamSavingEventArgs eventArgs = new StreamSavingEventArgs(zipEntry);\r\n OnSavingStream(eventArgs);\r\n \r\n //all other code that actually writes steam goes in here\r\n\r\n //note: reuse the eventargs so I can access my State object again\r\n OnSavedStream(eventArgs);\r\n}\r\n\r\n\r\nOne extra class:
\r\n\r\npublic class StreamSavingEventArgs : EventArgs {\r\n public StreamSavingEventArgs (ZipEntry zipEntry) {\r\n this.ZipEntry = zipEntry;\r\n }\r\n\r\n public object State { get; set; }\r\n\r\n public ZipEntry ZipEntry { get; private set; }\r\n}\r\n\r\n\r\nWith this I could do whatever I want:
\r\n\r\npublic void Run()\r\n{\r\n using (ZipFile zip = new ZipFile())\r\n {\r\n\t//add streams here\r\n\tzip.SavedStream += (o,e)=>e.ZipEntry.Stream.Dispose();\r\n\tzip.Save(ZipArchiveName);\r\n }\r\n}\r\n\r\n\r\nThe State property is optional but I can imagine a use where people would like to have an object in the end that they've created in the beginning.
\r\nYou could even make it more generic by creating SavingEntry and SavedEntry events that get called for each ZipEntry that's about to get saved or has just been saved. An enumerator could be used to check the type of entry.
\r\n\r\npublic void Run()\r\n{\r\n using (ZipFile zip = new ZipFile())\r\n {\r\n\t//add streams here\r\n\tzip.SavedEntry += (o,e)=>{if(e.ZipEntry.EntryType == ZipEntryType.Stream){e.ZipEntry.Stream.Dispose();}}\r\n\tzip.Save(ZipArchiveName);\r\n }\r\n}\r\n\r\n\r\nThat would probably be my choice. I don't know the library that well though and I don't know if it's hard to implement so I'm happy with your solution as is already.
\r\nThanks,
\r\nWes
\r\n", "PostedDate": "2009-03-05T01:54:45.847-08:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "164760", "ThreadId": "48865", "Html": "I would prefer something with delegates or events, but not an interface: using anonymous methods or lambdas is so easy.\r\n", "PostedDate": "2009-03-05T04:25:15.467-08:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "164875", "ThreadId": "48865", "Html": "Hey, thanks for the input.
\r\nyou know, I was halfway to implementing this feature with a matched pair of delegates, when I changed course and went to an interface. I had read something that said delegates are sort of an interface with a single method. And it felt not quite right to introduce 2 new delegates for this one new feature.
\r\nBut I can see your point.
There already is a SaveProgress event, and there's an emum that describes the "save state". Two of the states are: BeforeWriteEntry and AfterWriteEntry. These are the points in time when the library would need a stream to be opened and closed. At first I was thinking I could stuff this capability into that SaveProgress event, and even re-use those save states....but that didn't feel right either. Overloading the progress event to enable opening and closing streams... it felt like I was hiding the feature.
\r\nI could do this with a single new event and a new EventArgs type. Maybe call it StreamDispenserEventArgs. The point with that name is to distinguish it from the SaveProgressEventArgs that already exits. (SavingStreamEventArgs feels to similar). In the EventArgs type I could insert an enum that tells the app if the library is requesting a stream be opened, or closed (or whatever). And this event would be called not for every entry, but only for the entries with null (unspecified) streams. What do you think of that?
\r\n
\r\nor do you really want two distinct events here?
\r\n
\r\nNone of this is hard to implement in any case - the external interface is what we're talking about, and the internal mechanics are already there.
\r\n", "PostedDate": "2009-03-05T08:08:58.887-08:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "164928", "ThreadId": "48865", "Html": "Cheeso,
Wesley, what you say makes sense.
\r\njust one more thing -
\r\nthe name of the events. I get the rationale behind Verbing and Verbed. You proposed SavingStream and SavedStream, which makes sense to me if it is a geenral purpose event
But there is already a SaveProgress event, in which there is a BeforeSaveEntry and AfterSaveEntry. That event is a general purpose notification event.
\r\nOTOH, There's a distinct and more specific purpose for this pair of events we're discussing - to obtain and later dispose streams on a just-in-time basis. And I'm suggesting that the name should more clearly communicate that specific purpose. Also, to go along with that I'm suggesting that the operation and behavior also reflect the purpose - these events would be called only when needing a stream to be opened just-in-time, rather than being called for every entry being saved. In other words it's not really "SavingStream", it's "ObtainStream". We already have a something equivalent to SavingStream - it is in the SaveProgress event. By the same token, it's not really "SavedStream", it's "AllDoneWithYourStream".
\r\nWhat do you think?
\r\nThose are not the right names, but maybe you get my point: Saving and Saved seem not quite right.
\r\n", "PostedDate": "2009-03-05T09:45:04.017-08:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "164949", "ThreadId": "48865", "Html": "As soon as I wrote that I considered another usage scenario, from another thread on this forum, where the application was removing the file-to-be-zipped before it was completely zipped up.While browsing a little bit though the help file it's all getting a little bit clearer for me. Actually the functionally is in there allready IF the stream property of a ZipEntry could be reached.
\r\n\r\npublic static void SaveProgress(object sender, SaveProgressEventArgs e)\r\n{\r\n if (e.EventType == ZipProgressEventType.Saving_BeforeWriteEntry)\r\n {\r\n e.CurrentEntry.Stream = //Open your stream;\r\n }\r\n else if (e.EventType == ZipProgressEventType.Saving_AfterSaveEntry)\r\n {\r\n e.CurrentEntry.Stream.Dispose();\r\n }\r\n}\r\n\r\n\r\n\r\nSo maybe all you need to do is to make stream a reachable property.
\r\nCheers,
\r\nWes
\r\n", "PostedDate": "2009-03-05T10:39:05.217-08:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "165011", "ThreadId": "48865", "Html": "yes, exactly.
\r\nAnd by your making the suggestion, I am guessing you feel that such an extension does not overly complexify the SaveProgress event?
\r\n", "PostedDate": "2009-03-05T11:58:30.777-08:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "165047", "ThreadId": "48865", "Html": "No, I could definately work with this. It's just that I didn't expect the ZipEntry to be accessible from the progress event...\r\n", "PostedDate": "2009-03-05T14:10:38.917-08:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "165053", "ThreadId": "48865", "Html": "ok, give me a moment to run some tests and I will post the updated library with this change. \r\n", "PostedDate": "2009-03-05T14:16:01.03-08:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "165075", "ThreadId": "48865", "Html": "
Ok, the v1.8.1.11 library has the update where you can set the stream just-in-time.
\r\nTo use it you will code a SaveProgress event like so:
public static void SaveProgress(object sender, SaveProgressEventArgs e)\r\n {\r\n if (e.EventType == ZipProgressEventType.Saving_BeforeWriteEntry)\r\n {\r\n if (e.CurrentEntry.Source == Ionic.Zip.ZipEntry.EntrySource.Stream &&\r\n e.CurrentEntry.InputStream == null)\r\n {\r\n Console.WriteLine("opening {0}", e.CurrentEntry.FileName);\r\n System.IO.Stream s = MyStreamOpener(e.CurrentEntry.FileName);\r\n e.CurrentEntry.InputStream = s;\r\n }\r\n }\r\n else if (e.EventType == ZipProgressEventType.Saving_AfterWriteEntry)\r\n {\r\n if (e.CurrentEntry.InputStreamWasJitProvided)\r\n {\r\n Console.WriteLine("closing {0}", e.CurrentEntry.FileName);\r\n e.CurrentEntry.InputStream.Close();\r\n e.CurrentEntry.InputStream.Dispose();\r\n }\r\n }\r\n }\r\n\r\n
I put some extra properties in there to make it more convenient to use. It may be that people want to create a zip file using input from mixed sources - some from the filesystem and some from streams. In that case the library can provide some additional help to the application to help the app decide (a) whether it needs to open a stream in the BeforeWriteEntry event, and (b) whether it needs to Close/Dispose in the AfterWriteEntry event.
\r\n", "PostedDate": "2009-03-05T15:07:01.993-08:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "241293", "ThreadId": "48865", "Html": "For those who would miss the ZipOutputStream of SharpZipLib, here is a simple code that makes it possible to use DotNetZip the "regular .NET stream way".
\r\nHowever, take note that it is inefficient compared to a real on-the-fly streaming solution like SharpZipLib does, as it uses an internal MemoryStream before actually calling the DotNetZip.Save() function.
\r\nBut unfortunately, SharpZibLib does not sports EAS Encryption yet (and certainly never as the project seems abandonned now).
\r\nLet's hope that Cheeso will finally add this functionality in dotNetZip any time soon ? ;-)
\r\n\r\n
/// <summary>\r\n /// DotNetZip does not support streaming out-of-the-box up to version v1.8.\r\n /// This wrapper class helps doing so but unfortunately it has to use\r\n /// a temporary memory buffer internally which is quite inefficient\r\n /// (for instance, compared with the ZipOutputStream of SharpZibLib which has other drawbacks besides).\r\n /// </summary>\r\n public class DotNetZipOutputStream : Stream\r\n {\r\n public ZipFile ZipFile { get; private set; }\r\n\r\n private MemoryStream memStream = new MemoryStream();\r\n private String nextEntry = null;\r\n private Stream outputStream = null;\r\n private bool closed = false;\r\n\r\n public DotNetZipOutputStream(Stream baseOutputStream)\r\n {\r\n ZipFile = new ZipFile();\r\n outputStream = baseOutputStream;\r\n }\r\n\r\n public void PutNextEntry(String fileName)\r\n {\r\n memStream = new MemoryStream();\r\n nextEntry = fileName;\r\n }\r\n\r\n public override bool CanRead { get { return false; } }\r\n public override bool CanSeek { get { return false; } }\r\n public override bool CanWrite { get { return true; } }\r\n public override long Length { get { return memStream.Length; } }\r\n public override long Position\r\n {\r\n get { return memStream.Position; }\r\n set { memStream.Position = value; }\r\n }\r\n\r\n public override void Close()\r\n {\r\n if (closed) return;\r\n\r\n memStream.Position = 0;\r\n ZipFile.AddEntry(nextEntry, Path.GetDirectoryName(nextEntry), memStream);\r\n ZipFile.Save(outputStream);\r\n memStream.Close();\r\n closed = true;\r\n }\r\n\r\n public override void Flush()\r\n {\r\n memStream.Flush();\r\n }\r\n\r\n public override int Read(byte[] buffer, int offset, int count)\r\n {\r\n throw new NotSupportedException("Read");\r\n }\r\n\r\n public override long Seek(long offset, SeekOrigin origin)\r\n {\r\n throw new NotSupportedException("Seek");\r\n }\r\n\r\n public override void SetLength(long value)\r\n {\r\n memStream.SetLength(value);\r\n }\r\n\r\n public override void Write(byte[] buffer, int offset, int count)\r\n {\r\n memStream.Write(buffer, offset, count);\r\n }\r\n }\r\n\r\n
\r\n
\r\n
\r\n
> Let's hope that Cheeso will finally add this functionality in dotNetZip any time soon ? ;-)
\r\nWhat functionality is it, that you seek?
\r\nAre you asking for a different programming model? You don't like it that DotNetZip must always Read the file or stream for content, in order save a zip file?
\r\n> makes it possible to use DotNetZip the "regular .NET stream way".
\r\nI'm do not see a clear benefit from this approach. It's not a "regular stream". You have these extra methods - PutNextEntry() and so on, so a proposed DotNetZipOutputStream would not be usable in the same way any other stream is usable.
\r\nOne of the things I tried to do with DotNetZip even at the beginning is provide a metaphor that makes sense for zip files. That is the ZipFile class. Using a stream metaphor for the ZipFile seemed like a bad idea to me. It made the programming model more complicated. I still don't see the utility in it.
\r\nCan you explain it to me?
", "PostedDate": "2009-10-02T10:11:39.757-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "241422", "ThreadId": "48865", "Html": "I expect it's about an interface where the user can Write to a stream, not having the library Reading from a stream supplied by the user.
\r\nWhen the Save method is called, and when it is time to get the bytes for a given entry, it is also natural to let a callback just write the bytes (e.g. from a DataSet)
\r\nBut I find very aggressive to speak of DotNetZip that way; it is great, and is more than maintained. I thank you Cheeso for that.
", "PostedDate": "2009-10-02T11:20:38.557-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "241452", "ThreadId": "48865", "Html": "Thanks GM.
\r\nYes, I agree, the phrasing ("finally add this functionality") may seem a bit aggressive, but I didn't take it that way. I do try to maintain the library, and I haven't seen any workitems covering this idea. I know I am not dragging my feet on this.
\r\nWhat you say makes sense - allow the user to write to a stream that results in a ZipEntry in the zip file. I would be happy to discuss the idea to figure out a way to satisfy. In fact, just yesterday, someone came on the forums with this requirement. He wanted to save a DataSet as a zipentry in a zipfile, but he did not want to save the dataset to a file, first. The DataSet has a WriteXml() method, which writes to a stream. But DotNetZip cannot zip up the contents of a stream provided this way. My solution for that user was an AdapterStream. It works, but is not optimal. It requires the use of 2 threads, so the programming model is not so simple.
\r\nI think there may be a way to allow this, without modifying the rest of the DotNetZip interface too heavily. I'll have to think about it.
\r\nMy first idea is to have a callout from the library to the user's application, providing a stream to which the user must write. It might mean extending ZipEntrySource to include another option.
\r\nAnother idea is to introduce a new class in the way envisioned by the original poster. This would provide a completely alternative interface to DotNetZip. Everything would have to be duplicated - the events, all the ZipFile properties, and so on.
\r\n", "PostedDate": "2009-10-02T12:23:07.057-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "241480", "ThreadId": "48865", "Html": "
| \r\n I'm considering the programming model that would be enabled with this change. \r\n\r\n \r\nDataSet dataset1 = new DataSet();\r\ndataset1.Fill(...);\r\nusing (ZipOutputStream zipos = ZipFile.OpenOutputStream(stream))\r\n{\r\n zipos.CompressionLevel = Ionic.Zlib.CompressionLevel.Best; \r\n zipos.Encryption = Ionic.Zip.EncryptionAlgorithm.AES256;\r\n zipos.Password = "VerySecret!";\r\n\r\n zipos.PutNextEntry(...); \r\n dataset1.WriteXml(zipos); \r\n\r\n zipos.PutNextEntry(...); \r\n using (Stream otherStream = OpenReadableStream())\r\n {\r\n int n;\r\n while ((n= otherStream.Read(buffer, 0, buffer.Length))>0)\r\n {\r\n zipos.Write(buffer,0,n);\r\n }\r\n }\r\n}\r\n\r\n Is that about right? or...are you thinking something else? \r\nI suppose within PutNextEntry one would want to be able to set the encryption, compression level, codepage, and so on? Or, would it be ok to keep all of those properties on the ZipOutputStream class? \r\nI'm still trying to envision what it would look like. My first reaction is: such a ZipOutputStream class looks very redundant with the existing ZipFile class. I understand that it is similar to JarOutputStream in Java, and ZipOutputStream in SharpZipLib. But that fact alone is not a good reason to implement it in DotNetZip. \r\n\r\n \r\n | \r\n
| \r\n Or maybe a better interface is: \r\n\r\n \r\nusing (var zip = new ZipFile(output))\r\n{\r\n zip.Encryption= EncryptionAlgorithm.WinZipAes256;\r\n zip.Password = "whatever";\r\n\r\n using (ZipOutputStream zipos = zip.OpenOutputStream())\r\n {\r\n zipos.PutNextEntry(...); // name, other attributes \r\n dataset1.Write(zipos); \r\n\r\n zipos.PutNextEntry(...); // name, other attributes \r\n using (Stream otherStream = OpenReadableStream())\r\n {\r\n int n;\r\n while ((n= otherStream.Read(buffer, 0, buffer.Length))>0)\r\n {\r\n zipos.Write(buffer,0,n);\r\n }\r\n }\r\n }\r\n}\r\n\r\n In this case the ZipFile instance can dispense a ZipOutputStream, which can then be used as a regular, writable stream. Each PutNextEntry() starts a new entry. There is no Save() necessary on the ZipFile instance, because it would be written as the ZipOutputStream is written, and the zip file would be terminated when the ZipOutputStream is closed. \r\nAs an alternative I don't think a constructor like this: \r\n\r\n \r\nusing (Stream x = new ZipOutputStream(new ZipFile(output)))\r\n{\r\n ...\r\n}\r\n\r\n ...which inverts the construction, would make sense. The problem here is that the use of the ZipOutputStream changes the behavior of the ZipFile. Using ZipOutputStream means the zipfile is being written as calls to ZipOutputStream.Write() are made, which does not happen in the existing DotNetZip interface. Therefore, constructing a ZipOutputStream based on a ZipFile seems wrong. It would have to violate encapsulation in order to work properly. On the other hand, it seems ok to me, to change the state of the ZipFile, when using a method on the ZipFile to dispense a ZipOutputStream, as in the first code snip. \r\nCreating a ZipOutputStream is created from the ZipFile, and providing the expected semantics, would require some significant changes within the ZipFile class. While it wouldn't introduce redundancies with the existing interface, it would introduce some complexity and new pitfalls. \r\nFor example, intermixing calls to ZipFile.AddFile() with calls to ZipFile.OpenOutputStream() would introduce some twists. Using ZipFile.OpenOutputStream on a zipfile with entries in it, would cause all of the existing entries to be written/saved before the call returned. And so on. Nothing impossible to solve. \r\nAlso, is PutNextEntry really the right name for the method on the ZipOutputStream? How about PrepareNextEntry or StartNextEntry? PutNextEntry does not express what that method is doing. \r\n\r\n | \r\n
What would be convenient for me would be to have a constructor of ZipEntry (or a property on ZipEntry) to supply my own delagate of type Action<Stream> (or any other similar delegate if you don't want to depend on .Net 3.5).
\r\nThis delegate would do writes on the supplied stream, at the time the ZipEntry must be processed (inside the processing of ZipFile.Save).
\r\nYou have a Deflate implementation that would allow compression on the fly, and just have to pass to the delegate a Stream implementation that only allows Write (no reads, no seek) to the underlying file, and doing noting when closed/disposed.
", "PostedDate": "2009-10-03T05:50:48.42-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "241646", "ThreadId": "48865", "Html": "In fact I expect that internally you are reading a Stream only to write the bytes on a deflating stream(depending on CompressionMethod). Let us access that level. Reading comes from the use case of "copying" a file to the zip.
\r\nYou could internally supply a delegate for the existing cases, or let the user supply its own. The model of providing streams on an event, with another event to close it, is finally not convenient to use.
", "PostedDate": "2009-10-03T06:34:57.107-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "241709", "ThreadId": "48865", "Html": "yes, GM, you're right about the design. It reads the source stream only to write it to the DeflateStream.
\r\nI like the idea of a delegate, I'll have to think about modifying the design to allow that.
\r\nUp til now I have not depended on .NET 3.5. I don't know if I want to change that for DotNetZip v1.9.
", "PostedDate": "2009-10-03T12:52:52.2-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "241716", "ThreadId": "48865", "Html": "> The model of providing streams on an event, with another event to close it, is finally not convenient to use.
\r\nYou know, I thought about this for a while. How to provide a way to provide the stream just-in-time. I thought of providing 2 new delegate types - remember dotnetzip does not use .net 3.5, so I must define new types. This would mean one delegate type to open the stream and one delegate to close it. But then in the feedback I got, it seemed like too many new interfaces.
\r\nI could change it, but I am still not really happy with the delegate option for opening and closing the stream.
\r\nIf I provide a delegate option for writing the stream ... then it makes sense to also provide a delegate to open+close the stream. Those would be exclusive alternatives, of course. When the app provides a delegate to open the stream, it is DotNetZip calling Read on that stream (and never Write). On the other hand, when the app provides a delegate to Write the zip output stream, then DotNetZip wouldn't ever Read from an application-provided stream. Even though they are never used together, it would make sense to have them be consistent.
\r\n", "PostedDate": "2009-10-03T13:14:27.473-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "241754", "ThreadId": "48865", "Html": "
| \r\n Rather than providing a constructor, I think for consistency such an entry ought to be created with an overload of the AddEntry() method on the ZipFile class. \r\nThe new method should accept a delegate, which is defined like this: \r\n\r\n \r\npublic delegate void WriteDelegate(string entryName, System.IO.Stream stream);\r\n\r\n Here's the programming model, if you want to save a DataSet to a zip file. \r\n\r\n \r\nDataSet ds1 = new DataSet();\r\nda.Fill(ds1, "Invoices");\r\n\r\nusing(Ionic.Zip.ZipFile zip = new Ionic.Zip.ZipFile())\r\n{\r\n zip.AddEntry(zipEntryName, (name,stream) => ds1.WriteXml(stream) );\r\n zip.Save(zipFileName);\r\n}\r\n\r\n Or, for writing arbitrary data into the zip: \r\n\r\n \r\nusing (FileStream input = File.Open(filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite ))\r\n{\r\n using(Ionic.Zip.ZipFile zip = new Ionic.Zip.ZipFile())\r\n {\r\n zip.AddEntry(zipEntryName, (name,output) =>\r\n {\r\n byte[] buffer = new byte[BufferSize];\r\n int n;\r\n while ((n = input.Read(buffer, 0, buffer.Length)) != 0)\r\n {\r\n output.Write(buffer, 0, n);\r\n // could update a progress bar here\r\n }\r\n });\r\n\r\n zip.Save(zipFileName);\r\n }\r\n}\r\n\r\n\r\n\r\n What do you think? \r\n | \r\n
ok, I implemented this change, in changeset 44426.
\r\ndownload v1.9.0.12 to try it out.
\r\n", "PostedDate": "2009-10-03T23:52:17.76-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "242195", "ThreadId": "48865", "Html": "
First, I've got to apologize. Re-reading myself, I was definitly rude !
\r\nSorry. You make a wonderful job and I'm thankful for that.
\r\nTo clarify my point, I'm trying to use your library to receive and send XML data through a WCF service. Depending on metadata, the exchanged XML data are either compressed as a single-file zip archive or a deflated stream or just plain text.
\r\nUsing streams offers me to read and write the XML without worrying if it is compressed in a way or another or not at all. I'm just opening the correct chain of streams once at the beginning and then I can reuse the same path code that uses regular XmlTextReader/Writers to do the job. That's why the "stream" pattern suits me and why I'd like to use it with your library.
\r\nHowever, I'm facing two issues using streams this way:
\r\n- while writing, using the DotNetZipOutputStream, I'm running high on memory consumption as I need to hold the whole uncompressed data in memory before compressing.
\r\n- while reading, the MessageBobyStream received from the WCF interface is "forward-only" and the ZipFile.Read(stream) function is using the Seek() method which throws an UnsupportedException.
\r\nUnfortunately, your new AddEntry method with a delegate still does not fit with my needs, although it's great and a much better solution than the threaded StreamAdapter !.
\r\nCheers :-)
\r\n", "PostedDate": "2009-10-05T07:47:27.42-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "242206", "ThreadId": "48865", "Html": "
ok, that clarifies things.
\r\nWhy can you not save the zip file to your output stream? you have a stream, or a chain of streams, right? And you write to the outer-most stream., yes? why can you not just call ZipFile.Save(stream) ??
\r\nI don't really see the utility in sending a single-file zip archive. A deflated stream makes sense to me. Why would you want a single-file zip archive? It seems to offer no advantage over the DeflateStream approach.
\r\n", "PostedDate": "2009-10-05T08:15:26.903-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "242225", "ThreadId": "48865", "Html": "
The idea behind the single-zip archive thing is to be able to use the password protection with AES encryption to enforce the security the XML data stored on the client side before it is sent to the server. This is not directly related with the security of WCF services but I've got to deal with it at the other end as it is sent "as is".
\r\nAbout the ZipFile.Save(stream), it would mean that depending of the type of the source stream, I will have to pass the ZipFile object to my business layer that will have to know about it and whether it has to be called or not when it finishes its work. I would prefer that the business layer to deal only with raw XML data as a pivot, being unaware of encryption, authentication and protocols logic and details.
\r\n\r\n
", "PostedDate": "2009-10-05T08:49:52.207-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "242244", "ThreadId": "48865", "Html": "
| \r\n Ah, things are a little clearer. What you want is a stream that does Aes encryption. The ZipFile is not suitable for that purpose. \r\n> to enforce the security the XML data stored on the client side before it is sent to the server. This is not directly related with the security of WCF services but I've got to deal with it at the other end as it is sent "as is". \r\nI don't understand what this means. In WCF there is the possibility to encrypt the transmitted data on the sending side, and decrypt on the receiving side. Why not use the builtin capability? \r\n> I would prefer that the business layer to deal only with raw XML data as a pivot, being unaware of encryption, authentication and protocols logic and details. \r\nBut if you use the ZipOutputStream you provided above, your logic must call PutNextEntry() when writing to the zipOutputStream, right? And you must set the Password and the Encryption properties on the ZipFile. So is there knowledge of the ZipOutputStream in your code? There must be. \r\n\r\n | \r\n
boutblock, I think you want to do something like this:
\r\nusing (var input = File.Open(_inputFileName, FileMode.Open ))\r\n{\r\n using (var raw = File.Open(_outputFileName, FileMode.Create, FileAccess.ReadWrite ))\r\n {\r\n using (var output= new DotNetZipOneEntryOutputStream(raw, _inputFileName))\r\n {\r\n output.Password = _password;\r\n\r\n byte[] buffer= new byte[2048];\r\n int n;\r\n while ((n= input.Read(buffer,0,buffer.Length)) > 0)\r\n {\r\n output.Write(buffer,0,n);\r\n }\r\n }\r\n }\r\n}\r\n\r\n\r\n
is that right? if so, here's a class that will help you. It depends on the new feature added in v1.9.0.12. There's no MemoryStream used to buffer the writes. Instead it uses a background thread to do the save.
\r\n/// <summary>\r\n /// DotNetZip does not provide a ZipOutputStream, like SharpZipLib.\r\n /// This wrapper class provides one, which supports a single entry only.\r\n /// </summary>\r\n public class DotNetZipOneEntryOutputStream : Stream\r\n {\r\n private String _password;\r\n private String _entryName;\r\n private Stream _outputStream;\r\n private Stream _entryStream;\r\n private ZipEntry _zipEntry;\r\n private bool _closed;\r\n private System.Threading.ManualResetEvent _hasClosed;\r\n private System.Threading.ManualResetEvent _streamIsSet;\r\n private System.Threading.ManualResetEvent _doneSaving;\r\n\r\n /// <summary>\r\n /// Create a DotNetZipOneEntryOutputStream.\r\n /// </summary>\r\n ///\r\n /// <remarks>\r\n /// The resulting zip file will contain just one entry, with the specified\r\n /// name. \r\n /// </remarks>\r\n ///\r\n /// <param name="s">\r\n /// The stream to wrap.\r\n /// </param>\r\n ///\r\n /// <param name="entryName">\r\n /// The name of the entry to create in the zip file.\r\n /// </param>\r\n public DotNetZipOneEntryOutputStream(Stream s, string entryName)\r\n : this (s, entryName, null) { }\r\n\r\n \r\n /// <summary>\r\n /// Create a password-protected DotNetZipOneEntryOutputStream.\r\n /// </summary>\r\n ///\r\n /// <remarks>\r\n /// This class uses WinZipAes256 encryption when the password is\r\n /// non-null. The resulting zip file will contain just one\r\n /// entry, with the specified name, and encrypted using AES\r\n /// using the specified password.\r\n /// </remarks>\r\n ///\r\n /// <param name="s">\r\n /// The stream to wrap.\r\n /// </param>\r\n ///\r\n /// <param name="entryName">\r\n /// The name of the entry to create in the zip file.\r\n /// </param>\r\n ///\r\n /// <param name="password">\r\n /// The password to use to encrypt the entry in the zip file.\r\n /// </param>\r\n public DotNetZipOneEntryOutputStream(Stream s, string entryName, string password)\r\n {\r\n _hasClosed = new System.Threading.ManualResetEvent(false);\r\n _streamIsSet = new System.Threading.ManualResetEvent(false);\r\n _doneSaving = new System.Threading.ManualResetEvent(false);\r\n\r\n _entryName = entryName;\r\n _outputStream = s;\r\n _password = password;\r\n \r\n ThreadPool.QueueUserWorkItem(new WaitCallback(SaveWhenDoneWriting), null);\r\n _streamIsSet.WaitOne();\r\n }\r\n\r\n /// <summary>\r\n /// The password to use on the entry in the zip file.\r\n /// </summary>\r\n public string Password\r\n {\r\n set\r\n {\r\n _password = value; \r\n _zipEntry.Password = _password;\r\n _zipEntry.Encryption = (_password == null)\r\n ? EncryptionAlgorithm.None\r\n : EncryptionAlgorithm.WinZipAes256;\r\n }\r\n }\r\n\r\n /// <summary>\r\n /// The name of the entry in the zip file.\r\n /// </summary>\r\n public string EntryName\r\n {\r\n get \r\n {\r\n return _entryName;\r\n }\r\n set\r\n {\r\n if (value == null)\r\n throw new ArgumentException("EntryName");\r\n _entryName = value;\r\n _zipEntry.FileName = _entryName;\r\n }\r\n }\r\n\r\n \r\n private void SaveWhenDoneWriting(object o)\r\n {\r\n using (var zipFile = new ZipFile())\r\n {\r\n _zipEntry = zipFile.AddEntry(EntryName, (name, stream) =>\r\n {\r\n _entryStream = stream;\r\n _streamIsSet.Set();\r\n _hasClosed.WaitOne();\r\n }\r\n );\r\n if (_password != null)\r\n {\r\n _zipEntry.Password = _password;\r\n _zipEntry.Encryption = EncryptionAlgorithm.WinZipAes256;\r\n }\r\n \r\n zipFile.Save(_outputStream);\r\n }\r\n _doneSaving.Set();\r\n }\r\n\r\n /// <summary>\r\n /// Write the data from the buffer to the stream.\r\n /// </summary>\r\n public override void Write(byte[] buffer, int offset, int count)\r\n {\r\n if (_closed) throw new System.InvalidOperationException();\r\n _entryStream.Write(buffer,offset,count);\r\n }\r\n\r\n \r\n /// <summary>\r\n /// Close the stream. \r\n /// </summary>\r\n public override void Close()\r\n {\r\n if (_closed) return;\r\n _hasClosed.Set();\r\n _closed= true;\r\n _doneSaving.WaitOne();\r\n }\r\n\r\n\r\n public override bool CanRead { get { return false; } }\r\n public override bool CanSeek { get { return false; } }\r\n public override bool CanWrite { get { return true; } }\r\n public override long Length { get { throw new NotSupportedException(); }}\r\n\r\n public override long Position\r\n {\r\n get { throw new NotSupportedException();}\r\n set { throw new NotSupportedException();}\r\n }\r\n\r\n public override void Flush() { }\r\n\r\n public override int Read(byte[] buffer, int offset, int count)\r\n {\r\n throw new NotSupportedException("Read");\r\n }\r\n\r\n public override long Seek(long offset, SeekOrigin origin)\r\n {\r\n throw new NotSupportedException("Seek");\r\n }\r\n\r\n public override void SetLength(long value)\r\n {\r\n throw new NotSupportedException();\r\n }\r\n }\r\n\r\n\r\n\r\n
the XML data are persisted on the client side as zip archives to handle the case when the server is unavailable.
\r\nTherefore, the password and encryption for security reasons. WCF adds its own security over the streams exchanged.
\r\nAnd yes, I have to set the password, NextEntry and so on, but I do that during initialization of the chain of streams.
\r\nMy business layer only accept an open abstract Stream as a parameter (as unfortunately IStream does not exist in the framework...).
\r\nI will try your new DotNetZipOneEntryOutputStream as soon as ... I ll be back to work !
\r\nDon't you ever sleep or at least have a quick nap !! lol
\r\nNo, I don't sleep!!
\r\nI put a ZipOutputStream into DotNetZip - try it out. You need v1.9.0.13.
", "PostedDate": "2009-10-05T22:15:00.317-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "242469", "ThreadId": "48865", "Html": "Ok, I'm gonna try it now, thx a lot.
\r\nBtw, would be abusing to ask for a ZipInputStream that deals with forward-only streams ? ;-)
", "PostedDate": "2009-10-06T02:05:41.073-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "242481", "ThreadId": "48865", "Html": "Great ! I've replaced my own implementation based on a memory stream and it's working perfectly :-)
\r\nNow, If there is a way to workaround the forward-only input stream issue without using a temporary memory stream as I did, this would allow no limit in size of data exchanged by using streams in both directions of my WCF interface :-)
\r\nPS : my current version of Winrar 2.62 cannot open the zip archive with 256 bits AES encryption enabled (but hey, winrar is made for RAR not ZIP format ;-). However 7Zip 4.65 does perfectly.
\r\n\r\n
", "PostedDate": "2009-10-06T02:52:07.373-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "242788", "ThreadId": "48865", "Html": "
\r\nboutblock wrote:\r\n Are you referring to the fact that it is not possible to use PKZIP encryption when using a ZipOutputStream? If you are... \r\n\r\n\r\n If you are talking about something else, please explain. \r\n-Cheeso \r\n | \r\n
oh, wait, I think you are asking for a ZipInputStream, is that right?
\r\nYeah, I figured that would be the next step. I'll see what I can do.
\r\n", "PostedDate": "2009-10-06T21:15:29.327-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "242790", "ThreadId": "48865", "Html": "This discussion has been copied to a work item. Click here to go to the work item and continue the discussion.", "PostedDate": "2009-10-06T21:16:22.963-07:00", "UserRole": null, "MarkedAsAnswerDate": null }, { "Id": "242876", "ThreadId": "48865", "Html": "
Yeah, that's what I figured out about the legacy PKZIP encryption, that it wouldn't be compatible with a forward-only writing stream.
\r\nBut that's fine with me as the AES Encryption was what I needed in the first place.
\r\nI don't think you should bother with this old and weak PKZIP encryption while dealing with streams anyway.
\r\n\r\n
Regarding the ZipInputStream, well yes, that would make DotNetZip the most complete Zip library for .NET !
\r\nAnd I already have to thank you a lot for your responsiveness. You beat most of the commercial supports out there !
\r\n\r\n
", "PostedDate": "2009-10-07T02:15:33.28-07:00", "UserRole": null, "MarkedAsAnswerDate": null } ]