Does the ZIP format support all Unix file attributes and types, like hard links, named sockets and so on? I’m actually reluctant to use ZIP any more than we do currently, as the backup may not be complete…
1 Like
unzip command supports:
-K keep setuid/setgid/tacky permissions
[AtheOS, BeOS, Unix only] retain SUID/SGID/Tacky file attributes. Without this flag, these attribute bits are cleared for security reasons.
-X restore UID/GID info
[VMS, Unix, OS/2, NT, Tandem] restore owner/protection info (UICs and ACL entries) under VMS, or user and group info (UID/GID) under Unix, or access control lists (ACLs) under certain network-enabled versions of OS/2 (Warp Server with IBM LAN Server/Requester 3.0 to 5.0; Warp Connect with IBM Peer 1.0),
For links it’s not that promissing:
[Unix] Unix special files such as FIFO buffers (named pipes), block devices and character devices are not restored even if they are somehow represented in the zipfile, nor are hard-linked files relinked. Basically the only file types restored by unzip are regular files, directories and symbolic (soft) links.
I’m actually reluctant to use ZIP any more than we do currently, as the backup may not be complete
Expected and and very much understandable. But in this case scenario having zip as an option in the first place doesn’t make much of a help.
Windows did not have support for symbolic links at all in the OS until very recently, as far as I know. So…if you’re unzipping or untarring on anything older than Windows from the past couple years, you’re losing information.
Edit: I guess technically they’ve been there, but without tool support, for longer than that. But Windows 10 has proper support for symbolic links: Symlinks in Windows 10! - Windows Developer Blog
I don’t know what GNU tar or 7-zip or anything else does with them on Windows, though.
I think the main question here, if we should support compression format, which doesn’t compress data internally with user defined format (i.e. zip). I personally don’t like zip but if we have a zip option, I presume that data internally should also be packed using zip, otherwise it has little value, in my opinion.
Yeah, I’m surprised we offer zip. It’s not appropriate for the task, historically speaking.
1 Like
Yeah, I don’t recommend using ZIP at all with Virtualmin. However, some users do like it, so it’s there as an option. I would have to investigate if it’s safe to also use ZIP for sub-archives inside the backup though…
1 Like
@jimdunn Oh my, this is tiresome but I think you deserve an answer, more so as someone keeps reporting my posts, and the Virtualmin team keeps removing the flags…
Ah, yes, the conclusions… Mine is:
- you personally didn’t use the option, so this should be the standard and no one should; despite the necessity and my good intentions to just notify something that is, well, broken; WTH
- even if I propose all the decent scenarios, including removal of the option you try to make this into a militant thingy, like there is only one truth and such, option, path;
- you recommend the obvious which everyone told you it is not their case per se - most of us that have a server just use tar, mainly; incredibly in the Windowsphere people know about zip and not so much about tar - clients for example; and a widely spread free archiving app has issues with tar, so even a better reason to do something about it;
- let me sparkle your imagination with what can happen: you got a user, client, whatever > they end their relation with you, or just want to move, reset stuff whatever > you provide a backup of their data, or more so they take one, being at peace > it is a zip > you just deleted the domain, and until they realize something is wrong, your proper tar backups are already rotated > the zip is not a zip but also a tar (could get worse here, corruption wise) > and an archival software has already issues on its own > someone reports the situation > he gets shut by a bunch of accounts that pretty much is all they do;
- main question, you obviously didn’t test it, don’t use it and knew nothing/anything about the problem, why bother others then?
- so your contribution is mainly trying to muscle people around a properly reported bug, with the scope of… what? You are not providing data, nor testing, you are not providing help (didn’t ask for it BTW, not from you), you just waited to cling on a few words to enforce your… deleted position
So what’s your angle here, elevating your ego, because I just can’t see no good outcome, no contribution, nor proper technical intentions?
@staff , sorry for this post, in this thread I will not engage in any other comments, but as it seems that a few users have the right to just post flaming nonsense, I feel I am a little bit entitled, as the OP, to expose and condemn these posts as for what they truly are. Please close this thread, I am already sorry I brought this up…
Your report was correct and wanted. The Zip option did not do what it was supposed to do. Jamie and Ilia fixed the bug. This thread was productive and made Virtualmin better. We appreciate that.
The first post from Jim you responded angrily to was deleted quickly (by Jim, himself, I think, as I didn’t do it). What else do you want to happen? The next comment you’ve responded angrily to…your response seems overkill for what you’re responding to. It’s OK to disagree with someone (it’s OK for jimdunn and it’s OK for you). But, keep it civil.
Saying nothing is always an option for both of you. I don’t want to have to police everybody’s comments (though I’d rather do that than have every thread escalate into a flame war).
1 Like
Hello @Whoops
Like many others, I am highly motivated in helping people manage as much as possible of their digital life using Open Source software.
As per WinRAR - Wikipedia, WinRAR is not Open Source.
According to my research, 7-Zip and PeaZip are by far the two most popular solutions that
1- Run on Windows
2- Are Open Source
3- Are not discontinued
Ref: https://alternativeto.net/software/winrar/?license=opensource&platform=windows
I understand PeaZip re-uses some of the code of 7-Zip (A benefit of Open Source and collaboration) so they can demonstrate the same behavior in some circumstances.
I am confident this issue is now well identified and great progress is being made. If ever it was necessary to use a specific software, it should be documented in the UI.
While quick workarounds to solve a 1-time problem are nice, the goal here is to improve things for all current and future Virtualmin users.
Thanks!
2 Likes
@Jamie, we can approach the problem from another angle. I think the only reason why one would ever wanted to download as zip, as to quickly get all of the files for virtual server, and locally extract them with minimum hassle.
Therefore, I suggest simply to drop zip compression format option completely.
Instead, we should add a new mode (under backup destinations) or add a checkbox under modes select, which will provide a backup file that won’t be meant for restoration (we could emphasize it with ui_alert_box message). This kind of mode would produce either zip (if available) or tar.gz non-restorable backup file, without any internal compressions, i.e. databases would be found as dom_db1.sql, home directories would be found as directories and files just as files, so you can just extract it once and get all files accessible.
I think it would solve the problem.
In-fact we can and I think should replace Download immediately with this new mode, calling it something like Download plain via link.
Possible I suppose, but that would be a lot of work to implement and add a new backup format that we’d need to support. Better instead to use ZIP for internal archives if ZIP is being used for the primary compression …
I think I used 7-Zip a while back to extract locally, it didn’t work as intended, a switch to WinRar which solved my problem after reading other posts with similar issues. Initially, thought it was just me but later a few threads started to pop up. Few people extract locally, guessing that is why there haven’t been that many constructive reports, others probably switched compression utility and tried different things but without reporting back.
It isn’t about open source vs commercial/unsupported compression utility in use. It’s about Virtualmin UX and design behaviour of the options available, making it clearer, more specific in resolution the end user can take and remove options if not applicable. Making Virtualmin better is about providing constructive feedback. If all the noise is removed, there are some good posts in this thread and people can learn/share from that.
Hopefully, @fakemoth has had constructive feedback from the Virtualmin team which was the original intend of the first post I believe. I am sure others have found it useful too.
This sounds insanely complicated, both to implement and for an end user.
Can I voice in?
I’m with @Ilia in the idea of offering the zip format as a simple option for downloading the site’s files, and which cannot be used for restoration. A warning <div> in red that appears when “zip” is selected should hopefully be a simple workaround. And you don’t need to worry about storing symbolic links inside.
btw, Windows Vista was the first to support symbolic links, and I used to use them regularly on Win 7…
Thank you, guys 
One change I will make is to not compress files inside the ZIP archive in a different compression format, as it’s mostly redundant anyway (ie. MySQL dumps)
2 Likes
No, if that’s too difficult, it doesn’t worth an effort. Although, we don’t need to support that new backup format (it would be a format only for quick downloads).
Better instead to use ZIP for internal archives if ZIP is being used for the primary compression
Yes, that will work too and makes perfect sense.
However, I strongly suggest to drop an old mode Download immediately, as it’s not user friendly, especially on the large backups.
One change I will make is to not compress files inside the ZIP archive in a different compression format,
That worked with the latest patches, although _dir is still tar compressed when using zip format.
With the latest patches, the _dir file should be compressed with zip if the outer archive is zipped.
Confirmed - working.
@Jamie, last thing – is this possible to actually name the files inside of the archive as:
ubuntu18-gpl.local_dir.zip
ubuntu18-gpl.local_webmin.zip
...
or
ubuntu18-gpl.local_dir.tar
ubuntu18-gpl.local_webmin.tar
...
… so it would simply work out of the box without looking and renaming for files when extracted manually? It’s not difficult to fix the restore logic to ignore extension (.zip or .tar) on internal archive?
Good idea, I’ll look into this …
1 Like