Game:

The Spam Club

» The Spam Club - Life, The Universe and Everything - Software Galore - Disk Image dumps - Best practices?
ReplyNew TopicNew Poll

Disk Image dumps - Best practices?

Posted at 13:25 on November 11th, 2017 | Quote | Edit | Delete
Avatar
Member
Pupil Gumby
Posts: 14
START OF EDIT
*****************

Below you will find the results of my findings so far while looking into disk imaging and editing tools.

This list should only serve as a resource to get everyone started when it comes down to Disk Imaging or Editing tools. It is not complete nor everything has been 100% confirmed. Best practices differ for everyone, depending their preference, OS and disks you want to edit or dump.

Any feedback, tips or links to resources are MORE THAN WELCOME! :)

------------------------------------------------------------------------------------
DISK ANALYZING, EDITING & REPAIR ONLY TOOLS
------------------------------------------------------------------------------------

DISKEDIT 1.02 (Norton)
Manual: Included
Tools: OEM ID changing, Disk information, Hex editing, OEM ID changing
OS Support: DOS (works on NT Windows versions)

HXD
Manual: Included
Download: https://mh-nexus.de/en/downloads.php?product=HxD
Tools: OEM ID text editing
OS Support: Windows 95, 98, ME, NT 4, 2000, XP, 2003, Vista, or 7

WINHEX DISK EDITOR
Manual: Included
Download: http://www.winhex.com/disk-editor.html
Tools: OEM ID hex editing
OS Support: Windows 9x/ME

ANADISK
Manual: Included
Download: http://gd.tuwien.ac.at/pc/dos/msdos/diskutils/anad207.zip
Tools: Analysing, repairing disks, copying sector areas to files (not full disks).
OS support: UNCONFIRMED: DOS / Windows


--------------------------------------------------
OEM ID LIST
Here I'm just adding all IDs I come across on old disk and after formatting disks on different OS)
--------------------------------------------------

?????IHC - Windows 95 / 98 formatted or accessed disk
CSMAS3.0 -
IBM 3.3 -
MSDOS3.2 -
MSDOS3.3 - Stunts / 4D Sports Driving (5.25 '90)
MSDOS5.0 - Windows 2000 Formatted disk


------------------------------------------------------------------------------------
DISK IMAGING TOOLS
------------------------------------------------------------------------------------

WINIMAGE
Manual: http://www.winimage.com/wimushlp/
Download: http://www.winimage.com/download.htm
Formats: Raw or compressed
Input files: .IMA .IMG | UNCONFIRMED: .IMD .ISO .CIF .DSK .VUD .VFD .FLP .VMDK
Output files: .IMA | UNCONFIRMED: .FLP .IMZ .VFD
Copy protect.: -
Disk support: All 3.5" & 5.25" sizes
OS support: Windows users only, Windows 9X and NT based (up to Windows 10 64-bit)
Pros: Friendly interface, easy "Read Disk -> Save" method, good filetype compatibility, edit image OEM IDs
Cons: No copy protection capabilities

TIPS:
- Makes a perfect copy and does NOT change boot sector when using "Read Disk" then the "Save As" to uncompressed .IMA format.
- OEM ID can easily be edited in a uncompressed .IMA file without fear of damaging it because it's a raw format


TELEDISK
Manual: http://bitsavers.informatik.uni-stuttgart.de/pdf/sydex/Teledisk_1.05_Sep88.pdf
Download: http://www.classiccmp.org/dunfield/img54306/teledisk.htm
Format: undocumented compression, not Raw
Input file: .TD0
Output file: .TD0
Copy protect.: UNCONFIRMED: Basic copy protections (exact types PENDING TESTING)
Disk support: All 3.5 & 5.25 sizes
OS support: Real DOS (preferred), DOS shell in Windows 9x (NO 64-bit versions), DOSBox
Pros: Strong automated analysing tool, DOS user friendly interface, can copy basic protection
Cons: Limited file type support, no conversion

TIPS:
- When copying 360kB DS/DD disk you must choose DOS ONLY mode (not Both Sides) otherwise copying back doesn't seem to work (Error reading B: - Sector not found)
- No boot sector changes when writing back a (copy protection free) .TD0 image to disk.
- Changing the OEM ID in the .TD0 image file (with HxD) will disable the ability to write it back to disk


IMAGEDISK
Manual: Included
Download: http://www.classiccmp.org/dunfield/img/index.htm
Format: Raw
Input files: .IMD | UNCONFIRMED: .BIN .DMK .TD0
Outputs to: .IMD | UNCONFIRMED: .BIN .DMK .TD0
Copy protect.: UNCONFIRMED: Basic copy protections (exact types PENDING TESTING)
Disk support: All 3.5 & 5.25 sizes
OS support: Real DOS (preferred), DOS shell in Windows 9X (NO 64-bit versions, NO Windows 2000), DOSBox
Pros: Updated version of Teledisk, DOS user friendly interface, can copy basic protection, strong analyzing and custom settings tool, .BIN & .TD0 support
Cons: A tad more complex than TeleDisk, for advanced users

TIPS:
- Could not make it work on Windows 2000 ("No interrupt from FDC")
- Imagedisk stamp is added to boot sector in the image ONLY. However, copying the image back to disk results in the original bootsector without stamp
- Editing OEM ID in the image file and copying back to disk works perfectly
- Includes converters for IMD -> BIN and TD0 (Teledisk) -> IMD


DD
Manual: http://www.chrysocome.net/dd
Download: http://www.chrysocome.net/download
Format: Raw
Input files: .DD
Outputs to: .DD
Copy protect.: -
Disk support: All 3.5 & 5.25 sizes
OS support: Linux, NT based Windows only (Windows NT, 2000, XP, etc. NO Windows 9x or 64bit versions) | NO Real DOS / DOSBox
Pros: Works on Linux. Works in NT based Windows under a DOS shell
Cons: Limited file type support, no conversion, command based, for advanced users

TIPS:
- Works in Windows 2000 ("dd if=\\.\b: of=disk1" used)
- Works in Windows 2000 ("dd if=disk1 of=\\.\b:" used)
- Raw format, so editing image file and writing to floppy works


SAMDISK
Manual: http://simonowen.com/samdisk/options/
Download: http://simonowen.com/samdisk/
Formats: Raw / unconfirmed yet
Input files: UNCONFIRMED: .BPB .CFI .CPM .D2M .D4M .D80 .D81 .D88 .DFI .DMK .DS2 .DSC .DSK .DTI .EDSK .FDI .HFE .IMD .IPF .LIF .MBD .MGT .MSA .OPD .RAW .SAD .SBT .SCL .SCP .SDF .TD0 .TRD .UDI
Outputs to: UNCONFIRMED: .DSK .MGT .SAD .CPM .TRD .FDI .OPD .MBD .IMD .D81 .D88 .D2M .D4M .DTI .LIF .RAW
Copy protect.: UNCONFIRMED: Unknown (needs testing)
Disk support: UNCONFIRMED: all 3.5 & 5.25 sizes
OS support: Windows 9X and NT based (up to Windows 10 64-bit) | NO Real DOS / DOSBox
Pros: Best file type compability and conversion support.
Cons: Command based, for advanced users.

TIPS:
-

ULTRAISO
Manual: Included
Download: https://www.ezbsystems.com/ultraiso/download.htm
Format: Raw
Input files: .ISO | UNCONFIRMED: pending
Copy protect.: -
OS support: Windows 9x / NT / 64-bit versions
Pros: Adds virtual drive support
Cons: No support for virtual floppy drives

TIPS:
- Only handles 1.44 3.5" disks (would advise to use WinImage instead)
- Changing OEM ID in the .ISO file works and does not disable writing it back


RAWWRITE
Manual: Included
Download: http://www.chrysocome.net/download
In- and output format: Raw
Input files: .IMG
Output files: .IMG
Copy protect.: -
Disk support: ONLY 3.5" 1.44 MB
OS support: Windows users only, Windows 9X and NT based (up to Windows 10 64-bit)
Pros: Friendly interface
Cons: No copy protection capabilities, only supports 3.5 1.44MB disks

TIPS:
- Only handles 1.44 3.5" disks (would advise to use WinImage instead)
- Changing OEM ID in the .ISO file works and does not disable writing it back



-------------------------------------------------------------------
LIST OF KNOWN IMAGE FILE TYPES
-------------------------------------------------------------------

BPB - FAT12/16 BIOS Parameter Block
CFI - Compressed Floppy Image
CIF - Corel image format
CPM - Pro-DOS CP/M
D2M - Commodore CMD FD-2000
D4M - Commodore CMD FD-4000
D80 - Didaktik D80
D81 - Commodore 1581
D88 - Pasopeia D88
DFI - DiscFerret stream format
DMK - David M Keil's disk format
DS2 - Velesoft split side format
DSC - WinAPE disk image
DSK - Disk image
DTI - Deep Thought
EDSK - Extended disk imag
FDI - Full Disk Image
FLP - Virtual floppy image
HFE - SD HxC Floppy Emulator disk format
IMA - Windows based imaging software (WinImage)
IMD - ImageDisk utility image
IMG - Windows based imaging software (WinImage)
IMZ - Image file compressed (WinImage)
ISO - Standard CD based image
IPF - Interchangeable Preservation Format
LIF - Logical Interchange Format
MBD - MB-02+ Disk
MGT - MGT +D/Disciple/SAM
MSA - Magic Shadow Archive
OPD - OPus Discovery
RAW - KryoFlux stream format
*RAW - Raw dumps UNKOWN SOURCE
S24 - Sega System 24
SAD - SAm Disk
SBT - Sam BooTable disk
SCL - Sinclair betadisk archive
SCP - SuperCard Pro stream format
SDF - Sam Disk Format
TD0 - Sydex TeleDisk
TRD - Beta128 disk for TR-DOS
UDI - Ultra Disk Image
VFD - Virtual floppy disk image
VMDK - Virtual Machine Disk
VUD - Virtual Undo Disk

-------------------------------------------------------------------
LIST OF KNOWN COPY PROTECTIONS
-------------------------------------------------------------------

PENDING

-----------------------------------------------------------------------
LIST OF KNOWN COPY PROTECTED GAMES
-----------------------------------------------------------------------

PENDING


END OF EDIT
*********************

Hi all :)

Since I've started contributing games and disks I've started to delve in deeper into best practices for specifically disk dumping. After having read through quite a bunch of older threads here and (I think) all of the FAQs, I wanted to stitch together a quick Best Practice guide for dumping images 'correctly' for any newcomers (like me), depending on their situation (copy protection, disk size, operating system used, etc).

Would this be something valuable to pursue by the way? The FAQs gave me a very good start but did not help me all the way, which is why I come to you for your opinions and wisdom :)

Here's what I ran into so far on dumping images under Windows 98 / MS-DOS 6.22 with various software:
  • WinImage 8.1 2001 (WORKS) - Dumping my disks to images on my old WIN98 machine with WinImage 8.1 2001 seems to work like a charm. It outputs to .IMA which I afterward rename to .IMG. I haven't tried writing these WinImage dumps back to disk yet or mounting them, so they are unverified

  • DD (NOT WORKING) - I've tried using DD from within WIN98 in a DOS shell but I can't make it work properly as it keeps giving me errors (DD.EXE is located and run from the C: root, just to be sure). Commands I've tried:
    Quote:
    dd if=\\.\a: of=c:\disk.img -> "Error opening output file: 3 The system cannot find the file specified"

    Quote:
    dd if=a: of=c:\disk.img -> "Error opening output file: 5 Access is denied"

    Quote:
    dd id=a: of=c:\disk.img -> Crashes the program (ID was apparently added later as an "Input Device"
    command) with a "Could not find procedure FindFirstVolumeA" message

    Unfortunately, using "dd --list" does not work on WIN98 -> "--list is not available for Win95"
    I installed MS DOS 6.22 prior to WIN98 and tried using DD in the true DOS environment, but the program was mainly written for Windows

  • TELEDISK 2.16 (WORKS) - Used without issues to dump a 360K, 720K and 1.44MB disk. Yet to be verified!

Now for my actual questions, in the hopes someone will take the time and share their wisdom:
  • 1) Has anyone gotten DD to work properly under Windows 98 / DOS, or perhaps ran into the same errors I'm getting?

  • 2) Would it be better to use TELEDISK instead of WinImage 8.1 or try to get DD to work? I'm not sure if DD or WinImage 8.1 could handle bad sector copy protection

  • 3) If using TELEDISK, what version would be recommended to offer compatibility for 360K, 720K and 1.44MB disks and 40 tracks vs. 80 tracks compatibility? So far I've been using version 2.16 for all types (360K, 720K and 1.44MB) without issues, but haven't verified them yet.

  • 4) I've only noticed .IMA images in the archive so far, admittedly only having looked up a handful of games. If I use TELEDISK it will dump them to .TD0 files instead, or when using WinImage I use the .IMA extension (and rename them to .IMG). Is this OK or do you (admins) convert these back to an .IMG file afterwards?

Sorry for 'dumping' (ha!) all these questions on here, but hopefully someone can shed some wisdom with me so I don't have to reinvent the wheel :D

PS. I've also found very nice DOS programs that can be used to take screenshots in old DOS games on an actual old system. I'll add my findings here too if that seems useful, as some purists like me want to play, dump and review these games on old hardware.
-----
Edited by perfectnarcosis at 20:12 on November 26th, 2017
Posted at 13:34 on November 11th, 2017 | Quote | Edit | Delete
Member
Prof Gumby
Posts: 405
I don't recommend to use Winimage if your disk has error to dump.

I use normal disk dump tools on real DOS. (Just like Teledisk , ImageDisk)
Posted at 13:40 on November 11th, 2017 | Quote | Edit | Delete
Avatar
Admin
Reborn Gumby
Posts: 8517
PCXDump is usually the easiest to capture screenshots under MS-DOS. If that fails, Screenthief and Videothief are nice.
-----
Now you see the violence inherent in the system!
Posted at 13:53 on November 11th, 2017 | Quote | Edit | Delete
Avatar
Member
Pupil Gumby
Posts: 14
Thanks @ibmpc5150. I've seen your posts and replies on this subject in the other threads :) Is there an easy way to tell what game disks have this bad sector copy protection? If not, I will simply use TELEDISK from now on to be safe, and I'll give ImageDisk a go in DOS too.

Seeing however as the Disk Images FAQ explicitly mentions DD for Floppy Images and no other software, I'm going to assume this works on Windows NT/2000/XP/etc and simply not Windows 98? Might be useful to add that info on the FAQ or add the better TELEDISK / ImageDisk programs there too? Just a suggestion as it put me on the wrong path apparently :)

One question however still remains: it seems most images in the archive are .IMG? Is that mainly because people have dumped with ImageDisk or have most people been using WinImage, renamed it from .IMA to .IMG and ran into the possibility of uploading bad disks? Just a concern, since I'm also contributing these games as a secondary backup against future disk corruptions in my own collection.

@Mr Creosote: Thanks! I'll give PCXDump a go :) I used Screenthief for now and after messing with the settings I got it to output BMP files perfectly. I'll check PCXDump and compare a little to see which outputs best (size vs. quality vs. file type) and then stick to that.
-----
Edited by perfectnarcosis at 13:57 on November 11th, 2017
Posted at 14:25 on November 11th, 2017 | Quote | Edit | Delete
Avatar
Global Moderator
Retired Gumby
Posts: 861
Hello again!

First of all, thank you for your quick feedback as the latest floppy image dumping guidelines were created just a few days ago and are still pretty much raw. That is to say, it is all very relevant in the light of our latest activity, and we do, in fact, need some help in creating a lucid and reliable how-to.

Mind you, I'm not exactly an expert in this area, I only collect and organize files, so you will have to wait till our floppy manager arrives and provides you with some professional advice or perhaps for some other local users who happen to be somewhat up to date. However, I believe even I could help you with a thing or two.

Quote:
WinImage 8.1 2001 (WORKS) - Dumping my disks to images on my old WIN98 machine with WinImage 8.1 2001 seems to work like a charm. It outputs to .IMA which I afterward rename to .IMG. I haven't tried writing these WinImage dumps back to disk yet or mounting them, so they are unverified


One of the major problems with WinImage is that it leaves a hallmark in the file's boot record, which is unacceptable by our standards. Check the properties of your dumps and if the system label is WINIMAGE then it's pretty much a lost cause. I don't know if there is a way to make clean dumps, but generally speaking, WinImage is not recommended by us and is best ignored.

Quote:
DD (NOT WORKING) - I've tried using DD from within WIN98 in a DOS shell but I can't make it work properly as it keeps giving me errors (DD.EXE is located and run from the C: root, just to be sure). Commands I've tried:


Sorry, I don't mean to treat you like a noob, but are you sure your Win98 partition is healthy enough and is actually FAT32? Even though Win98 probably wouldn't run otherwise, but still, it's worth checking, as such issues are often caused by corrupt/incompatible FS.

Quote:
TELEDISK 2.16 (WORKS) - Used without issues to dump a 360K, 720K and 1.44MB disk. Yet to be verified!


We have certain problems regarding this particular app as it is equally beneficial and disconcerting. Its primary advantage is that it can dump images along with the copy protection scheme, which IMA/IMG and most other image formats are incapable of containing, yet it's poorly supported, incompatible with the majority of apps and is hardly processed. We do not exactly reject such files, but we can't publish them either. Not yet. It may have a future, but so far we treat this format as deprecated, and it's best to neglect it for now.

Quote:
or when using WinImage I use the .IMA extension (and rename them to .IMG). Is this OK or do you (admins) convert these back to an .IMG file afterwards?


Renaming alone is ok, we rarely convert anything. But do NOT use WinImage, please. Try UltraISO instead. I haven't got around to testing it myself yet, so, perhaps, you could give us all a hand. It's very flexible and user-friendly, try dumping some floppy with that and share the output, please.

Quote:
PS. I've also found very nice DOS programs that can be used to take screenshots in old DOS games on an actual old system. I'll add my findings here too if that seems useful, as some purists like me want to play, dump and review these games on old hardware.


It's a lot easier to do it in DOSBox nowadays and that hardly affects the image quality ;)

Quote:
Seeing however as the Disk Images FAQ explicitly mentions DD for Floppy Images and no other software, I'm going to assume this works on Windows NT/2000/XP/etc and simply not Windows 98?


Now that you say it I think the problem, could, in fact, lie in the missing NTFS, which is rather strange, but technically possible. I would suggest you use VirtualBox on your modern machine, install 2000 or XP there and try again.

Quote:
One question however still remains: it seems most images in the archive are .IMG? Is that mainly because people have dumped with ImageDisk or have most people been using WinImage, renamed it from .IMA to .IMG and ran into the possibility of uploading bad disks?


Like I said, we simply rename them to keep it homogeneous. Incidentally, lots of people upload bad dumps, but we still accept them for several reasons: 1) Potential backup for restoration, 2) For the lack of better options, 3) To prevent possible duplicates in the future. However, none of this means we request bad dumps, we only accept them as a matter of unintended mistake and ignorance ;)
-----
Cheer up! Remember the less you have, the more there is to get.
-----
Edited by Vagabond at 15:59 on November 11th, 2017
Posted at 17:13 on November 11th, 2017 | Quote | Edit | Delete
Avatar
Member
Pupil Gumby
Posts: 14
Originally posted by Vagabond at 14:25 on November 11th, 2017:
Hello again!

First of all, thank you for your quick feedback as the latest floppy image dumping guidelines were created just a few days ago and are still pretty much raw. That is to say, it is all very relevant in the light of our latest activity, and we do, in fact, need some help in creating a lucid and reliable how-to.

Mind you, I'm not exactly an expert in this area, I only collect and organize files, so you will have to wait till our floppy manager arrives and provides you with some professional advice or perhaps for some other local users who happen to be somewhat up to date. However, I believe even I could help you with a thing or two.


Happy to help out where I can, if I can :) Looking forward to the floppy manager and fire some of my questions so far, but a big thank you already for all the info you've supplied!

Quote:
Quote:
WinImage 8.1 2001 (WORKS) - Dumping my disks to images on my old WIN98 machine with WinImage 8.1 2001 seems to work like a charm. It outputs to .IMA which I afterward rename to .IMG. I haven't tried writing these WinImage dumps back to disk yet or mounting them, so they are unverified


One of the major problems with WinImage is that it leaves a hallmark in the file's boot record, which is unacceptable by our standards. Check the properties of your dumps and if the system label is WINIMAGE then it's pretty much a lost cause. I don't know if there is a way to make clean dumps, but generally speaking, WinImage is not recommended by us and is best ignored.


Ok, I've done quite a lot of reading and looking up old websites today (very soothing btw :D) for info on how OEM IDs work on Floppies throughout the course of MSDOS and Windows. There's a couple of things that I found out:
  • WinImage 8.1 does NOT change the OEM ID if you use the regular "Read Disk -> Save" method under Windows9X. I've tested this myself without any problems.

  • However, manually adding the floppy's contents into WinImage (click and drag into the WinImage window for example) and THEN using the Save option will simply create a new image, and since it was made by WinImage it automatically creates the OEM ID label "WINIMAGE". Thus the only proper way to use WinImage is through the "Read Disk -> Save" method. So far I've not run into problems on 8.5, but I'll try newer versions and on my modern machine to check for differences.

  • Last but not least, I found something worse. I found out that using any Windows9X (so NT/2000/XP or above are safe) to ACCESS ANY floppy that has its WRITE PROTECTION DISABLED will automatically result in Windows changing the OEM ID on this disk. The OEM ID is always changed to xxxxxIHC, where X is generated randomly, something most people think refers to the old Win95 codename "CHICAGO". The opinions on why this is done by Windows are different on the matter, but most think that it was implemented so that Windows could detect Floppy swaps more easily.

So, the following initial conclusions can cautiously be drawn:
- If you want to keep the original OEM ID on disks, NEVER EVER access them in WIN9X while their write protection is disabled! Unfortunately I did so on many of my disks and no longer know what their original IDs might be.
- It seems fine to use WinImage in the correct "Read Disk -> Save" way. NOT by manually adding files and creating an image afterwards.
- Just because the OEM ID mentions xxxxxxIHC does NOT mean the disk was wrongly dumped. The only thing we can conclude from this OEM ID is that the disk was (at one point in time) accessed on a Win9X OS with it's write protection disabled, and thus was overwritten by Windows.

That leaves however a couple of questions and situations to be cleared up:
- What is the impact of an 'unoriginal' OEM ID on these old disks, specifically with GoodOldDays' archive intentions in mind? Disks that have had ONLY their OEM IDs changed still work flawlessly when re-installing.

- So far I've only been able to find out that some DOS-formatted disks may depend on the OEM ID, like Novell/Caldera DOS 7.x disks. Additionally, reading non-DOS formatted disks like UNIX ones in Win9X can result in damaging them because UNIX may store critical info in the first sector, where the OEM ID is stored by Windows (see this link)

- Not crucial, but how can I check the OEM ID on images made by TELEDISK (.TD0)? I'd like to know so I can check if Teledisk also changes them or not. So far I've only been able to read OEM IDs from images and floppies directly with WinImage (Image -> Boot sector properties). If anyone knows an easier way let me know :)

- Considering the SHA-1 hash checksum is valued here, do SHA-1 hash checksums of disk dumps differ between a disk with its original OEM ID and with a changed OEM ID after accessing it in Win98?

- Do SHA-1 hash checksums differ between disks dumped by DD and when dumping the same disks with WinImage? Since I haven't been able to get DD to work on Win98 and all my disk reading hardware is on that old computer I won't be able to follow this check up anytime soon though.

Here's a thread I found on OEM IDs from DOS games that people have been gathering on a different platform: http://www.vcfed.org/forum/archive/index.php/t-36278.html
This might help, although I'm not sure how since I'm unsure of any additional impacts a changed OEM ID might have, other than not being classified as "original" anymore, which I think is debatable.

I'll try to spend some more time and gathering more facts. Feel free to point out any mistakes or post any follow up questions.
-----
Edited by Vagabond at 17:58 on November 11th, 2017
Posted at 18:47 on November 11th, 2017 | Quote | Edit | Delete
Avatar
Admin
Reborn Gumby
Posts: 8517
Simplified, I see three possible states of a disk image.

1. Perfect
2. Modified but working. Any modification, no matter how small, will lead to this. Including floppy label, modification timestamp etc. This is a matter of better than having nothing.
3. Bad, I.e. not working. Could be useful to collect nevertheless due to the reasons Vagabond stated.

The hashes will simply process the raw bitstream. Any modification, regardless of what it means, will result in a different hash. The hash function does not know what a disk label is, for instance. It will just see a different bit.

About the current FAQ, I wrote this one when I made the first dumps for the site. All information about MS Windows is second hand, as I have zero experience using it for anything beyond playing games on Windows 98.

Clearly, it is high time to amend it and we're working on it. Inputs more than welcome!
-----
Now you see the violence inherent in the system!
Posted at 20:50 on November 11th, 2017 | Quote | Edit | Delete
Avatar
Global Moderator
Retired Gumby
Posts: 861
perfectnarcosis, sorry, I had to edit your text sizing, because that way it was harder for us to read. If you don't mind too much, of course :)

Quote:
WinImage 8.1 does NOT change the OEM ID if you use the regular "Read Disk -> Save" method under Windows9X. I've tested this myself without any problems.


Ok, so there IS a way to make proper dumps using WinImage. Only the process is, evidently, a little too delicate, which is why I still wouldn't recommend it to the majority of people, unless they are really competent and know what they're doing.

Quote:
So far I've not run into problems on 8.5, but I'll try newer versions and on my modern machine to check for differences.


Please, do so and report back.

Quote:
The OEM ID is always changed to xxxxxIHC, where X is generated randomly, something most people think refers to the old Win95 codename "CHICAGO".


I've seen such labels actually, but I never seriously contemplated the origin of that. Very interesting observation.

Quote:
If you want to keep the original OEM ID on disks, NEVER EVER access them in WIN9X while their write protection is disabled! Unfortunately I did so on many of my disks and no longer know what their original IDs might be.


I believe we already have quite a few of such files in our archive, only they are considered eligible due to the issue being either noncrucial or simply unrecognized. Thanks for the heads-up.

Quote:
It seems fine to use WinImage in the correct "Read Disk -> Save" way. NOT by manually adding files and creating an image afterwards.


I guess I was a little hasty to pass the judgment, and this piece of info seems quite so reassuring. You know, you could actually prepare your own guidelines as to WinImage in particular to supplement our FAQ, for I believe there are few people out there who share the same degree of knowledge and experience when it comes right down to it. What do you say?

Quote:
That leaves however a couple of questions and situations to be cleared up:
- What is the impact of an 'unoriginal' OEM ID on these old disks, specifically with GoodOldDays' archive intentions in mind? Disks that have had ONLY their OEM IDs changed still work flawlessly when re-installing.


I believe I've already made that clear, we don't really care, at least not until now we haven't ;) But let's wait for our floppy manager to voice his opinion on this, for I still might be wrong. However, regardless of our opinion, we still accept all these files, so I'm not sure if it matters at all. Anyhow, if there is a working way to preserve the original ID's, then, of course, it's more than welcome. It's just that, alas, not everyone is as well-versed and pedantic as you, so it often leaves us no other choice but to take what is given, you see.

Quote:
Considering the SHA-1 hash checksum is valued here, do SHA-1 hash checksums of disk dumps differ between a disk with its original OEM ID and with a changed OEM ID after accessing it in Win98?


I believe a mere change of one single char in the file could immediately affect the checksum, which is what it's really intended for. And different OEM ID as a part of the file's interior is far more than just one char. So be sure, you can easily trace it ;)

Quote:
Do SHA-1 hash checksums differ between disks dumped by DD and when dumping the same disks with WinImage? Since I haven't been able to get DD to work on Win98 and all my disk reading hardware is on that old computer I won't be able to follow this check up anytime soon though.


Good question. It may actually differ this time, as these apps use different algorithms and dumping methods, and who knows what kind of small records and traces they leave behind. Only it doesn't matter as it's probably a minor discrepancy and should by no means affect the overall integrity. I do appreciate your scientific zeal, however, and I'm pretty sure you could answer all your questions yourself and be so very kind to update us, too :)

Quote:
This might help, although I'm not sure how since I'm unsure of any additional impacts a changed OEM ID might have, other than not being classified as "original" anymore, which I think is debatable.


Isn't there a way to edit this ID manually? Provided, of course, you have the original one.

Quote:
I'll try to spend some more time and gathering more facts. Feel free to point out any mistakes or post any follow up questions.


Something tells me it is you now who could point out our mistakes when it comes to true OCD and nazi dumping techniques. Just kidding ;) All your efforts and information are very much appreciated. Keep it up the good research.
-----
Cheer up! Remember the less you have, the more there is to get.
-----
Edited by Vagabond at 21:39 on November 11th, 2017
Posted at 02:03 on November 12th, 2017 | Quote | Edit | Delete
Avatar
Member
Pupil Gumby
Posts: 14
Quote:
perfectnarcosis, sorry, I had to edit your text sizing, because that way it was harder for us to read. If you don't mind too much, of course :)


@Vagabond: Haha, no worries man. I only realized later that it must have looked way too small for some, worse on Mobile. I'll stick to this size :D

Quote:
The hashes will simply process the raw bitstream. Any modification, regardless of what it means, will result in a different hash. The hash function does not know what a disk label is, for instance. It will just see a different bit.


@Creo: that's what I thought as well, thanks for confirming! I only wanted to know if the SHA-1 hash was of any more importance to you in some other way, since it's explicitly listed next to the disk images. Perhaps adding the OEM ID of the image in a 3rd column on the disk images list would be an idea worth looking into? It could potentially tell us more and point out potential differences between disks.

Thanks for clarifying on all points by the way, I felt it was important for me to understand what you guys value in the dumps and get to know your structure :)

Quote:
I guess I was a little hasty to pass the judgment, and this piece of info seems quite so reassuring. You know, you could actually prepare your own guidelines as to WinImage in particular to supplement our FAQ, for I believe there are few people out there who share the same degree of knowledge and experience when it comes right down to it. What do you say?


Sure thing! For whoever's interested, found this thread which contains quite a few lists of copy protected games, Teledisk's creator posting some cool stuff and a lot of WinImage mentions. In particular that WinImage handles all non-protected games very well, but cannot handle bad byte protection so instructions for it should be pretty straight forward.

Teledisk or Anadisk (same creator) however seem to have been popping up regularly in my recent findings as best practice for handling all disk types, recommended in a real DOS environment, including best for copy protected games. The thread linked above touches upon that a few times, so have a read if you're interested in knowing more. They also talk about the hardest to beat copy protection there is, intentional physical damage to disks haha.

Quote:
Isn't there a way to edit this ID manually? Provided, of course, you have the original one.


Yep, it's easily edited but indeed, if you don't know the original ID then it's lost. One way to find it again is by checking other disk images found on the web, but depends on the format stored. I currently only know how to read the OEM IDs by using WinImage and the filetypes that supports. But no, losing the ID is no biggie for the usefulness of the disk's file content. Falling back to the "MSDOS" ID is an option mentioned here.

I'll keep you all posted on further results on WinImage, DD, UltraISO and other image tools that I've been picking up on along the way. Installing Windows 2000 and Windows XP now on a second drive, but still on the old machine so I've got access to all floppy drives and should be able to get a quick assessment.
Posted at 08:40 on November 12th, 2017 | Quote | Edit | Delete
Avatar
Global Moderator
Retired Gumby
Posts: 861
Quote:
that's what I thought as well, thanks for confirming! I only wanted to know if the SHA-1 hash was of any more importance to you in some other way, since it's explicitly listed next to the disk images. Perhaps adding the OEM ID of the image in a 3rd column on the disk images list would be an idea worth looking into? It could potentially tell us more and point out potential differences between disks.


We specify such details in the comments part and set appropriate status, in this case likely "Modified". I've seen some of the last comments made by our floppy manager where he mentions OEM ID specifically, but apparently it's not a common practice for him, because I don't think I have seen him do it before. What I'm getting at, is that now lots of our images will have to be either marked as "Modified" and have a respective note attached or somehow restored.

Quote:
Yep, it's easily edited but indeed, if you don't know the original ID then it's lost. One way to find it again is by checking other disk images found on the web, but depends on the format stored.


Or, perhaps, find someone who owns the real floppies and ask them kindly to share the ID. Do you suppose it would be too much to ask for? ;)
-----
Cheer up! Remember the less you have, the more there is to get.
Posted at 21:28 on November 14th, 2017 | Quote | Edit | Delete
Avatar
Member
Pupil Gumby
Posts: 14
Quote:
Or, perhaps, find someone who owns the real floppies and ask them kindly to share the ID. Do you suppose it would be too much to ask for? ;)


Hehe, absolutely does not hurt! I've already started doing so and have re-edited some of my disks that had gotten overwritten already ;)

Here's a short update on OEM ID reading/editing tools that I came across:
  • WinImage can only read a physical floppy's OEM ID, but not edit it directly. It can however change an disk image's OEM ID. So just to be clear there is no way of editing a real floppy's OEM ID with it on the fly, bummer! :(

  • For Windows NT or newer I recommend WinHex Disk Editor (does not run on Windows 9X). This program will read a floppy disk and present the OEM ID quite nicely on-screen, but still requires manually selecting the ID in the HEX code and change it. A tad risky for inexperienced users.

  • Windows9X users can either use HXD (hex editor) or default back to a DOS program (which is what I did).

  • Best and easiest DOS program I found is DISKEDIT (Norton Utilities). This runs in a real DOS env. but also in a DOS shell in Windows. Least amount of risk involved because it will read a floppy and display the OEM ID in a separate text field, which you can edit without running the risk of editing anything else. A tad slow on Windows OS, but seems the least risky one for inexperienced users. Program has its own menus and navigation, to make it easier.

  • NEVER EVER set your disks to writable if you care about your OEM IDs and are using a Win9x system to view/install/play ;) Can't iterate this enough

Last but not least, an update on Disk Imaging Tools for different OS and copy protections. The more I dive into this the more complex it seems to become haha :o But it's been interesting so far:
  • WinImage 8.1 or newer - Does not change OEM IDs when handled correctly (as mentioned before). Confirmed to work nicely on Win9X, 2000 and XP. Easiest to use for undamaged and unprotected disks. Outputs a full 360/720/1.44 image by default (does not skip empty sectors).

  • DD - Linux / Unix based users, but also compatible with Win2000 and up (needs NTFS support to work). Commandline based and works best for undamaged and unprotected disks.

  • Teledisk (manual) / ImageDisk / SAMdisk - All DOS based but can be run in a shell. All seem to offer similar compatibility in terms of copy protection, but these three can at least handle basic form of copy protection pretty well.

  • Hardware devices that can handle more extreme forms of copy protection are SuperCard Pro, KryoFlux, or DiscFerret. These can read the attached floppy on a very deep level and copy it practically 1 on 1 into an image
Please take these findings with a grain of salt. If anyone reading this has feedback let me know :)

Next steps
I'm currently in the process of getting some new old stock 5.25 floppies (360 and 720), as well as 3.5 floppies (720/1.44) to further test some low end copy protection and compare the disk imaging tools further, in particular to see if writing the images back works for unprotected and (minimal) protected disks with DD, Teledisk, ImageDisk, SamDisk and WinImage.

As I mentioned above, any copy protection that is more complex than Teledisk or ImageDisk can handle (I have NOT find a master list of protection types yet..), can only be done with external hardware like KryoFlux. According to what I've read and found so far, even then it can be tricky to get these KryoFlux raw images back onto a disk and work on a real system, depending on the protection method. The best way to still make use of the dumped images in this case is apparently through emulation (PCmE), which of course still makes it worth backing up the data here, among other reasons :)
Posted at 01:00 on November 15th, 2017 | Quote | Edit | Delete
Avatar
Global Moderator
Retired Gumby
Posts: 861
Quote:
WinImage can only read a physical floppy's OEM ID, but not edit it directly. It can however change an disk image's OEM ID. So just to be clear there is no way of editing a real floppy's OEM ID with it on the fly, bummer! :(


What if you dump an image, fix it and then copy it back?

Quote:
Best and easiest DOS program I found is DISKEDIT (Norton Utilities). This runs in a real DOS env. but also in a DOS shell in Windows


Does that include DOSBox? I noticed you never refer to it for some reason, as though you don't trust it enough ;) I mean, I kind of gathered that you prefer to use original operating systems onboard a matching hardware, but you do realize that most people have to resort to emulation most of the time, don't you? At least both I and Wandrell do, let alone a bunch of other people on here. So, if we were to use these tools we would likely attempt to do it in DOSBox, VirtualBox and similar software.

Quote:
NEVER EVER set your disks to writable if you care about your OEM IDs and are using a Win9x system to view/install/play


Too bad so many users out there are probably ignorant of this which results in so many modified images, even if slightly, which is, in fact, only an aftermath of improperly handled physical media in the past, which isn't their fault but, strictly speaking, inappropriate behavior on part of OS. Good thing it can still be corrected, though!

Quote:
DD - Linux / Unix based users, but also compatible with Win2000 and up (needs NTFS support to work). Commandline based and works best for undamaged and unprotected disks.


See, I told you it must have something to do with FS ;) Only it's strange. Why would it matter...

Quote:
I'm currently in the process of getting some new old stock 5.25 floppies (360 and 720), as well as 3.5 floppies (720/1.44) to further test some low end copy protection and compare the disk imaging tools further, in particular to see if writing the images back works for unprotected and (minimal) protected disks with DD, Teledisk, ImageDisk, SamDisk and WinImage.


Looking forward to your reports!
-----
Cheer up! Remember the less you have, the more there is to get.
Posted at 06:46 on November 15th, 2017 | Quote | Edit | Delete
Avatar
Admin
Reborn Gumby
Posts: 8517
perfectnarcosis, thank you so much for your in-depth work so far! Could you elaborate a little on file format support of each tool? Especially, is each bound to one format each? What, in your view, are the pros and cons of each format? You briefly mentioned the issue of writing back Kyroflux.
-----
Now you see the violence inherent in the system!
Posted at 21:04 on November 15th, 2017 | Quote | Edit | Delete
Avatar
Member
Pupil Gumby
Posts: 14
Quote:
What if you dump an image, fix it and then copy it back?

@Vagabond: Yep, that should work indeed. It's also something I'd like to test when I've got those floppies I'm waiting for.

Quote:
Does that include DOSBox? I noticed you never refer to it for some reason, as though you don't trust it enough ;) I mean, I kind of gathered that you prefer to use original operating systems onboard a matching hardware, but you do realize that most people have to resort to emulation most of the time, don't you? At least both I and Wandrell do, let alone a bunch of other people on here. So, if we were to use these tools we would likely attempt to do it in DOSBox, VirtualBox and similar software.

Haha, I love DOSBox actually. But for some reason when I started gathering old hardware about a year ago and build this 1999 rig, I've started to do everything 'old school' lol. But sure thing, I've used DOSBox quite often in the past, so I'll definitely include it when testing for all these tools' compatibility.

Quote:
Could you elaborate a little on file format support of each tool? Especially, is each bound to one format each? What, in your view, are the pros and cons of each format? You briefly mentioned the issue of writing back Kyroflux.

@Mr Creosote: Very good point :) I have been looking at their file formats too and found several lists with some info on them, but so far it's all small bits and pieces I'm afraid. Cheers for reminding me however!

I'm thinking about editing the initial post with the findings so far, would that be alright?
Posted at 17:18 on November 19th, 2017 | Quote | Edit | Delete
Avatar
Member
Pupil Gumby
Posts: 14
Spend another good few hours today and last night gathering a more complete picture of the Tools mentioned so far.

I've edited my initial post with these results, but I seem to keep spending way too much time formatting everything back to readable BB CODE. Do we support HTML in any way or form? Or does anyone know of a better way to deal with tables on the Forum here?

In a nutshell: I've collected a lot of additional information (and confirmed where I could on the fly) about the most spoken off tools out there, their OS support, file type support, some brief Pros and Cons on each, where to find the program or extended manuals.

I've also started a list of known image file types, copy protection types and games that have known copy protection. I will keep updating my initial post while I progress in confirming all these points.

@all: feel free (I encourage you! :D) to help out in confirming features of any of these tools if you seem to use them, share your knowledge of known copy protections, etc. Any help is appreciated :)

-perfectnarcosis
-----
Edited by perfectnarcosis at 17:21 on November 19th, 2017
Posted at 14:21 on November 20th, 2017 | Quote | Edit | Delete
Avatar
Admin
Reborn Gumby
Posts: 8517
Looks good so far! Word of caution: let's not confuse a file name extension with a file format. Since we are talking about preservation here, the most important question for file formats for me is whether it is openly documented and free of patent and other claims.

When I last checked SPS, for example, they claimed their format would not be understood by anyone in the world and therefore, they would not release documentation. Which goes against the very idea of preservation, in my view - no matter how good their software and hardware may be.
-----
Now you see the violence inherent in the system!
Posted at 20:30 on November 26th, 2017 | Quote | Edit | Delete
Avatar
Member
Pupil Gumby
Posts: 14
@Mr Creosote: you're absolutely right. I've changed the sections and added the tools' main output format, where I could find out. Most output to a raw format, but some use compression or undocumented formats.

After spending more time this weekend, here's what I've edited in the initial post again:
  • Started a list of OEM IDs I come across or linked to certain OS formats
  • Clarified format vs. filenames
  • Confirmed all disk support sections
  • Confirmed all tools' principal in- and output files and formats
  • Added RawWrite as a possible tool for (modern) Windows users
  • Added some tips for each tool I've used so far

It's been rather informative going through each tool's copy and write back processes, and comparing the differences of their outputs (boot sector mainly). Of all outputted images, only TeleDisk and ImageDisk formats (TD0 and IMD) seemed to have changed the bootsector when comparing these to the original disk. However, writing back these images to disk did NOT result in these changes remaining. It appears they are only added to the images and removed once written back to a disk. This might give us some more insight in how to judge images that have been dumped and submitted :)

For next week I'm going to focus on Copy Protection, known types/games and hopefully I may have a game in my possession with basic copy protection that Teledisk or ImageDisk are supposed to be able to handle. If not I'll try to write an image back to disk that has known copy protection, and work from there.
Posted at 19:23 on December 28th, 2017 | Quote | Edit | Delete
Member
Baby Gumby
Posts: 3
Thank you for this exhaustive list of all the programs and what they can do. I have a large library of floppies 3.5/5.25 that I need to archive. Some I have no idea what is actually on them, some are unreadable in my DOS machine. Again, thank you!
ReplyNew TopicNew Poll
Powered by Spam Board 5.2.4 © 2007 - 2011 Spam Board Team