File size on disk bigger than expected with file stream
Hello,
we use a NAS to archive stream from database to files. If we look at the file size on disk, it is bigger than expected.
For example :
File size is 127K
ls -alh | grep write-chupmb-pdf 127K 2020-07-02 10:43 write-chupmb-pdf
Size on disk is 192K
du -sh * | grep write-chupmb-pdf 192K write-chupmb-pdf
Or expected size is 128K (with 4K blocs)
I tried with %Stream.FileBinary, %FileBinaryStream and OPEN/USE/WRITE and with multiple files and file type (PDF, JPG, PNG) and I have the same problem each time.
For example :
du -sh * | grep write-chupmb-pdf 192K write-chupmb-pdf 192K write-chupmb-pdf-copyfrom 224K write-chupmb-pdf-raw 192K write-chupmb-pdf-tmpstream
And the code to generate these files
//stream variable come from database (%GlobalBinaryStream) //write-chupmb-pdf-copyfrom SET fileNAS = ##class(%Stream.FileBinary).%New() DO fileNAS.FilenameSet("\\nas-chup-7d\TSarchives\2018\TEST\write-chupmb-pdf-copyfrom") WRITE fileNAS.CopyFromAndSave(stream),! DO stream.Rewind() //write-chupmb-pdf SET fileNAS = ##class(%Stream.FileBinary).%New() DO fileNAS.FilenameSet("\\nas-chup-7d\TSarchives\2018\TEST\write-chupmb-pdf") WHILE 'stream.AtEnd { DO fileNAS.Write(stream.Read()) } WRITE fileNAS.%Save(),! DO stream.Rewind() //write-chupmb-pdf-tmpstream SET tmpStream = ##class(%Stream.GlobalBinary).%New() DO tmpStream.CopyFrom(stream) SET fileNAS = ##class(%Stream.FileBinary).%New() DO fileNAS.FilenameSet("\\nas-chup-7d\TSarchives\2018\TEST\write-chupmb-pdf-tmpstream") WRITE fileNAS.CopyFromAndSave(stream),! DO stream.Rewind() // write-chupmb-pdf-raw SET fileNASRAW = "\\nas-chup-7d\TSarchives\2018\TEST\write-chupmb-pdf-raw" OPEN fileNASRAW:("WSN") USE fileNASRAW WHILE 'stream.AtEnd { WRITE stream.Read() } CLOSE fileNASRAW
I also compared with streams written by other technologies than Caché and we have no problem with them.
Has anyone ever encountered this kind of problem ?
Caché version is : Cache for Windows (x86-64) 2017.2.2 (Build 865U) Mon Jun 25 2018 10:33:28 EDT
Thank you
Yann
What's your file I/O default?
zn "%SYS" do ^NLS Choose 4) Display loaded settings Choose 3) Display system defaults for I/O tables
Here's what I got:
-------------- System defaults for I/O tables -------------- Table Name ----------------- --------------- Process RAW Cache Terminal UTF8 Other terminal UTF8 File UTF8 Magtape UTF8 TCP/IP RAW System call RAW Printer CP1251
Here's what I got
Thank you
As a rule of thumb disks allocate files in blocks larger than the I/O block. This is called the allocation size or cluster size. I believe the default for large Windows disks is 64 kio. This page has some explanation: <https://docs.microsoft.com/en-us/Exchange/plan-and-deploy/deployment-ref..., search for NTFS allocation unit size.