Encoding, Encryption, and Hashing

Encoding, Encryption, and Hashing Encoding is often confused with encryption and hashing. They are not the same. But before I go into the differences, I'll first mention the similarities: All three transform data into another format. Both encoding and encryption are reversible, unlike hashing.

Encoding The purpose of encoding is to transform data so that it can be properly (and safely) consumed by a different type of system, e.g. binary data being sent over email, or viewing special characters on a web page. The goal is not to keep information secret, but rather to ensure that it's able to be properly consumed. Encoding transforms data into another format using a scheme that is publicly available so that it can easily be reversed. It does not require a key as the only thing required to decode it is the algorithm that was used to encode it. Examples: ASCII, Unicode, URL Encoding, Base64

Encryption The purpose of encryption is to transform data in order to keep it secret from others, e.g. sending someone a secret letter that only they should be able to read, or securely sending a password over the Internet. Rather than focusing on usability, the goal is to ensure the data cannot be consumed by anyone other than the intended recipient(s). Encryption transforms data into another format in such a way that only specific individual(s) can reverse the transformation. It uses a key, which is kept secret, in conjunction with the plaintext and the algorithm, in order to perform the encryption operation. As such, the ciphertext, algorithm, and key are all required to return to the plaintext. Examples: AES, Blowfish, RSA

Hashing Hashing serves the purpose of ensuring integrity, i.e. making it so that if something is changed you can know that it's changed. Technically, hashing takes arbitrary input and produce a fixed-length string that has the following attributes: The same input will always produce the same output. Multiple disparate inputs should not produce the same output. It should not be possible to go from the output to the input. Any modification of a given input should result in drastic change to the hash.

Hashing Hashing is used in conjunction with authentication to produce strong evidence that a given message has not been modified. This is accomplished by taking a given input, hashing it, and then encrypting the sent hash with the recipient's public key. When the recipient opens the message with their private key they then hash the message themselves and compare it to the hash that was given encrypted by the sender. If they match it is an unmodified message. Examples: SHA-3, MD5 (Now obsolete), etc.

Hash collision Collision or clash is a situation that occurs when two distinct pieces of data have the same hash value , checksum , fingerprint , or cryptographic digest. When hash functions and fingerprints are used to identify similar data, such as homologous DNA sequences or similar audio files, the functions are designed so as to maximize the probability of collision between distinct but similar data. Checksums, on the other hand, are designed to minimize the probability of collisions between similar inputs, without regard for collisions between very different inputs.

Hash Collision Attack A Collision Attack is an attempt to find two input strings of a hash function that produce the same hash result. Because hash functions have infinite input length and a predefined output length, there is inevitably going to be the possibility of two different inputs that produce the same output hash. If two separate inputs produce the same hash output, it is called a collision. This collision can then be exploited by any application that compares two hashes together – such as password hashes, file integrity checks, etc.

MD5 MD5 is an algorithm that is used to verify data integrity through the creation of a 128-bit message digest from data input (which may be a message of any length) that is claimed to be as unique to that specific data as a fingerprint is to the specific individual. MD5, which was developed by Professor Ronald L. Rivest of MIT, is intended for use with digital signature applications, which require that large files must be compressed by a secure method before being encrypted with a secret key, under a public key cryptosystem. MD5 is currently a standard, Internet Engineering Task Force (IETF) Request for Comments (RFC) 1321.

Hex editor A hex editor (or binary file editor or byte editor) is a type of computer program that allows for manipulation of the fundamental binary data that constitutes a computer file. The name 'hex' comes from 'hexadecimal': the standard numerical format for editing binary data. A typical computer file occupies multiple areas on the platter(s) of a disk drive, whose contents are combined to form the file. Hex editors that were designed to parse and edit sector data from the physical segments of floppy or hard disks were sometimes called sector editors or disk editors.

Bit rot Bit rot is often defined as the event in which the small electric charge of a bit in memory disperses, possibly altering program code or stored data. The hypothesis that semiconductor RAM may occasionally be altered by cosmic rays is also known as soft error. Bit rot can also be used to describe the phenomenon of storage media gradually decaying over the duration of many years.

Decay of storage media Solid state media — such as EPROMs, flash memory and other solid-state drives— stores data using electrical charges, which can slowly leak away due to imperfect insulation. The chip itself is not affected by this, so re-programming it once per decade or so will prevent bit rot. The biggest problem can be finding a clean copy of the chip from which to make the copy; frequently, by the time the user discovers the bit rot, there are no un-damaged chips to use as a master. Magnetic media — such as Floppy disk and magnetic tape — may experience bit rot as bits lose their magnetic orientation. Also, in warm and humid conditions these media are prone to literal rotting.

Decay of storage media Optical media — such as CD-R, DVD-R and BD-R — may experience bit rot from the breakdown of the material onto which the data is stored. This can be mitigated by storing discs in a dark, cool location with low humidity. "Archival quality" discs are also available, but do not necessarily provide a permanent solution to the onset of bit rot or other types of data corruption beyond a certain amount of time. Some media (such as M-DISC) are designed to improve longevity over DVD-R and BD-R. Paper media — such as punched cards and punched tape — may also experience literal rotting. Mylar punched tape is available for use in this situation.