CRC

CRC_RevEng is a fantastic tool for this, if you can figure out how to use it.  Hopefully, my text, code, and videos below can help you do this.

The base case for my reverse engineering a CRC for a QIC format so far was the Kennedy 64xx format, which was by far the most challenging.  There was no polynomial listed in the documentation, and the block size was HUGE, and varied in length.

Just simply running CRC_RevEng against many thousands of unique message/CRC samples all returned "no models found", without exception.

So, the over-simplified explanation about how I derived these was that I first wrote a program that wrote a batch file which tried every possible 16-bit polynomial against a single message/CRC sample...with 65536 different reveng commands, and only one of them actually found a result: 0x8005.

Then I wrote another program that wrote another batch file to run reveng to check ~1200 unique message/CRC samples using only polynomial 0x8005, and I got these results shown above.

(And what did I mean by...
[Append-Initialization-In-Message Value Search]

???  This must be the result set of the 2nd program, not the first...)

---------------------------------------------------------------------------------------

So, here's the exhaustive verbose detail:

One of the misconceptions I had about CRC_RevEng, is that when you call a file from the command line, that the file must be in binary format, not a text file with the hexadecimal characters represented.  What threw me was all of the examples showed text-based hexadecimal characters as a parameter passed in the command line, so I wrongly assumed, that a file that it was to open from the command line, using the -f switch, should also be text.  WRONG...it must be binary.  I found that out by posting on the CCTalk group and made this video to explain it.  https://youtu.be/nH-tfbvXYrI  

If you really want to understand the full struggle of this process, please watch this video:

I had addressed the video to Greg, the maker of CRC-RevEng, but he never acknowledged that he watched it.  As it turns out, a CCTalk group contributor helped me figure this out...Lawrence.  (Thanks again, Lawrence!).  You can see that message thread here:  http://bit.ly/1MXMfvg

After knowing that, I then posted a job on Upwork.com, with this job description:

We need someone to write a simple program, in any language (the simpler the better), that converts a text-string of hexadecimal characters into a binary file.

This video demonstrates exactly what needs to be done:

https://youtu.be/yCCz0TojuaY

Please watch this video, and make your application.

We hope this is a very simple task for you, and should take only a few minutes.

Thanks, and we look forward to your application.

Well, I got a result within hours.  An Upwork programmer Ivan Kovalenko 

( https://www.upwork.com/freelancers/~01ccc9342132038d28 ) wrote a perfect Ruby script for this, and it works beautifully:

#!/usr/bin/ruby

if $[ 0 ].nil?() or not FileTest.exists?( $[ 0 ] ) then

    puts( "Usage: #{ $0 } <input-file>" )

    exit( 1 )

end

i = 1

File.open( $*[ 0 ], 'r' ) { | input |

    while not input.eof?() do

        line = input.readline().chomp()

        next if line.empty?()

        File.open( "#{ $*[ 0 ].gsub( /\..+$/, '' ) }-#{ sprintf( '%06d', i ) }.bin", 'wb' ) { | output |

            output.write( [ line ].pack( 'H*' ) )

        }

        i += 1

    end

}

Program One:  The Polynomial Hunter

So, then, I used this program to run in my program, to turn my text string blocks of hexadecimal characters into a true binary file.  I'm going to attempt to explain that program in this video here 

[being created very soon]

Here's the LotusScript key sub-routine of my FIRST program that does the bulk of the work:

Sub CRCRevEng

BinaryConversionProgram = "C:\Ruby21\bin\ruby C:\Data\MightyFrame\CRC\Ruby\hex2bin.rb " + OutputFileName8 ' use for Ruby program

' BinaryConversionProgram = "C:\Data\MightyFrame\CRC\VB\Hex2Bin.exe " + OutputFileName8 ' User for VB Program

taskId% = Shell (BinaryConversionProgram,6)

CRCBinaryFileName1 = OutputPath + ASourceFileNameWithoutSpaces + "-S"

Filler = "C:\X\X-S-000001.bin C:\X\X-S-000001.bin C:\X\X-S-000002.bin C:\X\X-S-000003.bin C:\X\X-S-000004.bin C:\X\X-S-000005.bin C:\X\X-S-000006.bin C:\X\X-S-000007.bin C:\X\X-S-000008.bin C:\X\X-S-000009.bin"

For poly = 1 To 65535

RevEngParams = "-w 16 -p " + Hex(poly) + " -s -f"

CRCRevCheckFile = Filler  ' Use for Ruby program

CRCRevCheckCommand = "reveng " + RevEngParams + " " + CRCRevCheckFile 

Print #9, CRCRevCheckCommand 

Next poly

End Sub

.........................

Program Two:  The Initialization Prepend Hunter