{ "WorkItem": { "AffectedComponent": { "Name": "", "DisplayName": "" }, "ClosedComment": "", "ClosedDate": null, "CommentCount": 0, "Custom": null, "Description": "The attached file when run through the following code (C++ WinForms app with single button & click handler):\n \n\tprivate: System::Void button1_Click(System::Object^ sender, System::EventArgs^ e) \n\t\t{\n\t\t\ttry\n\t\t\t{\n\t\t\t\tIonic::Zip::ZipFile^ zipArchive = gcnew Ionic::Zip::ZipFile;\n\t\t\t\tzipArchive->Encryption = Ionic::Zip::EncryptionAlgorithm::WinZipAes256;\n\t\t\t\tzipArchive->Password = L\"a\";\n\t\t\t\tzipArchive->UseZip64WhenSaving = Ionic::Zip::Zip64Option::AsNecessary;\n\t\t\t\tzipArchive->CompressionLevel = Ionic::Zlib::CompressionLevel::BestSpeed;\n\t\t\t\tzipArchive->Strategy = Ionic::Zlib::CompressionStrategy::HuffmanOnly;\n\t\t\t\tzipArchive->AddDirectory(\"G:\\\\ziptest\\\\source\");\n\t\t\t\tzipArchive->Save(\"G:\\\\ziptest\\\\test.zip\");\n \n\t\t\t\tIonic::Zip::ZipFile^ zip = Ionic::Zip::ZipFile::Read(\"G:\\\\ziptest\\\\test.zip\");\n\t\t\t\tzip->Password = L\"a\";\n\t\t\t\tzip->ExtractAll(\"G:\\\\ziptest\\\\output\", Ionic::Zip::ExtractExistingFileAction::OverwriteSilently);\n\t\t\t}\n\t\t\tcatch(Exception^ ex)\n\t\t\t{\n\t\t\t\tMessageBox::Show(ex->Message);\n\t\t\t}\n\t\t}\n \nresults in a BadStateException with the error \"The final hash has not been computed\"\n \nThe test.zip generated is 295167 bytes.\n \nWhen stepping through the decryption process, the final state of the WinZipAesCipherStream shows _length as 294915, and _totalBytesXFerred as 294912 (a 16k boundary)\n \nThe final 16k block retrieved from the Aes stream is Deflated such that the ZlibCodec goes from AvailableBytesIn = 16k, AvailableBytesOut = 32k to AvailableBytesIn = 689 AvailableBytesOut =0.\n \nZipEntry::_ExtractOne then requests the penultimate 32k of data: AvailableBytesIn =689, AvailableBytesOut = 32k goes to AvailableBytesIn = 4 AvailableBytesOut = 0\nZipEntry::_ExtractOne then requests the final 3932 bytes: AvailableBytesIn = 4, AvailableBytesOut = 3932 goes to AvailableBytesIn = 0, AvailableBytesOut = 0\n \nSo the ZipEntry believes it has read all the data, but somehow the last 3 bytes are never read from the aes stream, so the hmac is not computed.\n \nI'm afraid that is as far as I got with my analysis, as the deflate algorithm is somewhat fiendish to step through!\n \nI hope this is enough for you to go on. I have seen in some circumstances this file encrypts to about 344k, then decrypts fine. I can't quite put my finger on when it encrypts/decrypts ok, it seems to depend on the number of other files in the archive (the original had several hundred) and password, though this may seem pretty unlikely!", "LastUpdatedDate": "2013-02-21T18:43:23.863-08:00", "PlannedForRelease": "", "ReleaseVisibleToPublic": false, "Priority": { "Name": "Low", "Severity": 50, "Id": 1 }, "ProjectName": "DotNetZip", "ReportedDate": "2010-10-25T10:21:25.623-07:00", "Status": { "Name": "Proposed", "Id": 1 }, "ReasonClosed": { "Name": "Unassigned" }, "Summary": "Decryption of file results in \"The final hash has not been computed\" error", "Type": { "Name": "Issue", "Id": 3 }, "VoteCount": 2, "Id": 12361 }, "FileAttachments": [ { "FileId": 3582, "FileName": "IMG203", "DownloadUrl": ".\\3582" }, { "FileId": 3583, "FileName": "test.zip", "DownloadUrl": ".\\3583" } ], "Comments": [ { "Message": "Since I have seen the original file work ok, here's the zip file as it gets generated on my machine, in case for some reason the code above won't recreate the issue...", "PostedDate": "2010-10-25T10:24:14.747-07:00", "Id": -2147483648 }, { "Message": "Hi Cheeso,\r\nHave you had any luck recreating this issue? Let me know if you need more info...", "PostedDate": "2010-11-22T03:26:15.117-08:00", "Id": -2147483648 }, { "Message": "Hi Bobby,\r\n\r\nI think your problem might be related to the thread here: http://dotnetzip.codeplex.com/workitem/10562\r\n\r\nThe main loop in the *compression* routine sometimes exits too early if it lands halfway across writing the adler value. In this case it's written the entire compressed file data so it wrongly quits without writing the rest of the pending adler bytes (which the 3 bytes you're missing above are part of).\r\n\r\nThe nature of the problem means it only happens when you hit something like (the size of the compressed file) mod (the size of the write buffer) <= 4 , which is why it only occasionally fails.\r\n\r\nI posted some unofficial fixes in that thread which might solve your problem - it's a bit of a jumble but let me know if you have trouble applying them.\r\n\r\nCheers,\r\n\r\nMike", "PostedDate": "2010-11-24T13:55:04.4-08:00", "Id": -2147483648 }, { "Message": "Hi Mike, \r\n\r\nThanks for pointing me at 10562 - the issue certainly looks related, with the compressed size % 16k < 4. I implemented the fixes you suggest, however they didn't fix my problem.\r\n\r\nInterestingly I have the issue with decompression (I think I have muddled 'Deflate' and 'Inflate' terminology in earlier posts) rather than compression - 7zip and WinRar can open the zip just fine. \r\n\r\nI stepped through the compression code out of interest to see what effect your changes had made, but interestingly my app is using a plain FileStream to read the original and ParallelDeflateOutputStream to write the zip rather than ZLibBaseStream for the compression, so the changes are never hit.\r\n\r\nWhen I come to decompress the zip, it is using ZLibBaseStream - but the problem seems to be in the InflateManager.Inflate() method - the changes from your thread are relating to the 'nomoreinput' flag in the ZLibBaseStream, but this never gets set in my example as the AvailableBytesIn reaches 0 at the same point as AvailableBytesOut reaches 0 which one way or another leads to the WinZipAesCipherStream _finalBlock flag not being set and the hmac not being computed.\r\n\r\nI will investigate more closely to see if I can work out whether the Inflated data is correct but the flags are not being set, or whether the Inflate algorithm itself has an issue.", "PostedDate": "2010-11-25T10:44:42.48-08:00", "Id": -2147483648 }, { "Message": "Hi Bobby,\r\n\r\nI can reproduce your problem here although I get a completely different zip file - it's the same size, but \"fc /b\" lists a whole load of binary differences. I get the same error on decompression though.\r\n\r\nInterestingly, if you set \"zipArchive.ParallelDeflateThreshold = -1\" on the compression side then it decompresses just fine.\r\n\r\nI'm going to have a little play around and see if I can find anything else out...\r\n\r\nM\r\n", "PostedDate": "2010-11-25T13:13:32.053-08:00", "Id": -2147483648 }, { "Message": "Hi Mike,\r\nI think the binary files will differ each time you aes encrypt them, as there is a random 'salt' applied to the encryption algorithm (http://www.winzip.com/aes_info.htm#salt) to prevent attacks where known content in one encrypted file could be detected in a different encrypted file allowing reverse engineering of the key, or something.\r\n\r\nInteresting about the deflate threashold though - I have just tried it and -1 works for me too. The resulting zip is around 346k - about the same size as when I have seen it working before (see last paragraph of original post).\r\n\r\nI don't know if this indicates a possible issue in the parallel deflate, or whether the result is just different (as is indicated in the help page for parallel deflate - the dictionaries used are different in the parallel method, so the result will be different; though the help suggests the parallel method would usually be bigger- we're seeing the opposite).\r\n\r\nOr maybe the different size of resulting object means the size is not within 4 of a 16k threashold, hence the decompression issue just goes away, as it does with 3999 out of 4000 (or is that 4095 out of 4096!) archives.\r\n\r\nI'm still intrigued that other apps can open the zips produced by either method, which suggests that the encryption part is just fine but there's something going on in the decryption/inflation process. Will keep looking!", "PostedDate": "2010-11-26T02:26:54.247-08:00", "Id": -2147483648 }, { "Message": "If I add a break point in the finaly clause of ZipEntry.InternalExatract, the file on disk after output.Close() and before File.Delete(TargetFile) is exactly correct - so the final 3 bytes left unread in the encryption stream must be some form of padding (or checksum / hash?) not required by the Inflate process, hence the outer streams believe they have finished, as they have all the data they require, but the innermost AES stream is left with 3 bytes still to read, hence it has not computed the final hmac, and hence the exception.\r\n\r\nI'm wondering whether the generic answer is that in the AES stream FinalAuthentication() method, rather than throwing an exception if the finalblock flag is not set, should read to the end of the input regardless of the current state at this point, as it seems that no other point in the process knows that these bytes may still be waiting, and their only relevance is for the internal consistency of the encrypted data hash value.\r\n\r\nI'm trying to think of cases where a genuine error would cause the breaking out of the encrypted stream read before the end - but the purpose of the hmac is to ensure integrity of the encrypted data - if the outer streams think they have got the data they require, then the CRC check will say if the decrypted data is consistent, so I think the encrypted stream can assume that if there are bytes left to read, they are wholey irrelevant to the encapsuated object, and only relevant to the encryption process itself.", "PostedDate": "2010-11-26T07:38:04.91-08:00", "Id": -2147483648 }, { "Message": "...So I've altered WinZipAes.cs FinalAuthentication() as follows, which works with my example image - any thoughts on the validity of this anyone?!\r\n\r\n public byte[] FinalAuthentication\r\n {\r\n get\r\n {\r\n if ( _totalBytesXferred != 0)\r\n {\r\n byte[] buffer = new byte[16384];\r\n while (!_finalBlock)\r\n {\r\n long bytesRemaining = _length - _totalBytesXferred;\r\n int count = (bytesRemaining > buffer.Length) ? buffer.Length : (int)bytesRemaining;\r\n\r\n _s.Read(buffer, 0, count);\r\n TransformInPlace(buffer, 0, count);\r\n _totalBytesXferred += count;\r\n }\r\n// throw new BadStateException(\"The final hash has not been computed.\");\r\n }\r\n else\r\n {\r\n // special-case zero-byte files\r\n // Must call ComputeHash on an empty byte array when no data\r\n // has run through the MAC.\r\n byte[] b = { };\r\n _mac.ComputeHash(b);\r\n }\r\n byte[] macBytes10 = new byte[10];\r\n System.Array.Copy(_mac.Hash, 0, macBytes10, 0, 10);\r\n return macBytes10;\r\n }\r\n }", "PostedDate": "2010-11-26T07:39:25.86-08:00", "Id": -2147483648 }, { "Message": "I've done a bit of digging and ended up in roughly the same place, but from a slightly different angle...\r\n\r\nIn the single-threaded version, the call to FinalAuthentication has _finalBlock==true so it skips that section entirely. In the parallel version _finalBlock==false in FinalAuthentication, which triggers the whole exception path. I think the solution might be hidden somewhere in the calling loop and making it perform another iteration to empty its pipeline (setting _finalBlock=true in the process) before it calls FinalAuthentication.\r\n\r\nYour fix sort of does, but I'm going to keep digging to see if it can be solved in the calling loop...\r\n\r\nI'll let you know how I get on.\r\n\r\nM", "PostedDate": "2010-11-26T13:14:18.687-08:00", "Id": -2147483648 }, { "Message": "Hi Bobby,\r\n\r\nI've spent a fair bit of time looking at this, but I can't quite work out where those 3 extra bytes come from in the encrypted stream. I made some pretty big changes to the WinZipAes.cs code to try and track it down, but my version still has the same problem...\r\n\r\nI'm quite interested in trying to get to the bottom of this so I'm going to keep plugging on. I'll let you know if anything turns up.\r\n\r\nM", "PostedDate": "2010-12-06T16:52:35.227-08:00", "Id": -2147483648 }, { "Message": "Hi Bobby,\r\n\r\nHere's an update I've made to GetFinalAuthentication - it's got a bit more error handling in it. A couple of variable names have changed as well, but it should be obvious what the old ones are to change them back.\r\n\r\nLet me know if you've got any thoughts about it.\r\n\r\nCheers,\r\n\r\nMike\r\n\r\n /// \r\n /// Returns the final HMAC-SHA1-80 for the data that was encrypted.\r\n /// \r\n public override byte[] GetFinalAuthentication()\r\n {\r\n if (!this.finalBlock)\r\n {\r\n // special-case zero-byte files\r\n if (this.totalBytesRead == 0)\r\n {\r\n // Must call ComputeHash on an empty byte array when no data\r\n // has run through the MAC.\r\n this._mac.ComputeHash(new byte[] {});\r\n }\r\n else\r\n {\r\n // some AES-encrypted zip entries have a number of padding bytes\r\n // after the main file data (e.g files created using\r\n // Zlib.ParallelDeflateOutputStream). WinZip and other Zip readers\r\n // seem to be able to handle these so we'll do the same here. The\r\n // padding data needs to be sent through the MAC calculator, but\r\n // we can discard the decrypted data.\r\n while (!this.finalBlock)\r\n {\r\n if (this.totalBytesRead == this.dataLength)\r\n {\r\n // we've read past all the file data including any padding,\r\n // but we've still not found the final block\r\n throw new Ionic.Zip.BadStateException(\"The final hash has not been computed.\");\r\n }\r\n // there's still some more padding data in the encrypted\r\n // zip data, so read some of it now\r\n byte[] buffer = new byte[16384];\r\n int bytesFromStream = this.Read(buffer, 0, buffer.Length);\r\n if (bytesFromStream == 0)\r\n {\r\n // nothing could be read from the stream.\r\n // maybe it closed itself or something...\r\n throw new System.IO.IOException(\"The stream could not be read.\");\r\n }\r\n }\r\n }\r\n }\r\n byte[] macBytes10 = new byte[10];\r\n System.Array.Copy(this._mac.Hash, 0, macBytes10, 0, 10);\r\n return macBytes10;\r\n }", "PostedDate": "2010-12-07T15:47:08.8-08:00", "Id": -2147483648 }, { "Message": "Hi Mike,\r\nSorry for the delay in responding - have been called onto other projects for the last month or two... With the addition of the error checking I think the proposed solutions work - my only thought being that you have the check for 0 byte files inside the !finalBlock check. I'm not sure what the finalBlock state would be for a zero length file - but to be on the safe side should the zero length file check be outside the !finalBlock check?\r\n\r\nOther than that my only reservation is that we are 'doing stuff' in the property getter, which is considered by some to be bad style, but I'm not sure there is an easy alternative. Thanks for your help on this!\r\nB", "PostedDate": "2011-02-08T10:45:44.603-08:00", "Id": -2147483648 }, { "Message": "Hi Bobby,\r\n\r\nNo problem - you're welcome.\r\n\r\nAs it happens, I'm not totally sure which way round the two tests should go either - I think I tried to replicate the original code as closely as possible, but it's been a while since I looked at it so I could be wrong. Sorry that's not much help!\r\n\r\nOn a separate note, I did just spot a possible bug in my last post - the \"int bytesFromStream = this.Read(buffer, 0, buffer.Length);\" line *could* end up reading past the end of the entry. It should probably be something like this:\r\n\r\nint bytesFromStream = this.Read(buffer, 0, Math.Min(buffer.Length, this.dataLength - this.totalBytesRead);\r\n\r\nso the read gets truncated to the end of the data for the entry. (Then again, \"this.Read\" might do that for us internally so it may not be needed).\r\n\r\nAs for the property accessor issue - I changed it to a method in my last post. I seem to remember the convention is to prefix potentially long-running tasks with \"Get\" so I plopped that on the front of the name as well.\r\n\r\nHopefully we've cracked this one now :-)\r\n\r\nM", "PostedDate": "2011-02-10T12:37:43.157-08:00", "Id": -2147483648 }, { "Message": "", "PostedDate": "2011-02-11T08:11:50.253-08:00", "Id": -2147483648 }, { "Message": "I have the same problem. Any solution for this issue?", "PostedDate": "2012-05-16T02:54:19.167-07:00", "Id": -2147483648 }, { "Message": "", "PostedDate": "2013-02-21T18:43:23.863-08:00", "Id": -2147483648 } ] }