Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » Developing U++ » UppHub » Encrypted storage with streaming (OpenSSL, AES)
Re: Encrypted storage with streaming (OpenSSL, AES) [message #25402 is a reply to message #25401] Mon, 22 February 2010 08:31 Go to previous messageGo to next message
Mindtraveller is currently offline  Mindtraveller
Messages: 917
Registered: August 2007
Location: Russia, Moscow rgn.
Experienced Contributor

koldo wrote on Mon, 22 February 2010 09:54

1) Does it mean that AES cannot be used for saving user files with user defined password ?

However there are programs that include this possibility with AES. For example 7zip offers AES-256 encryption http://www.7-zip.org/7z.html.

2) Is there a standard way to convert a 8 chars user defined password into an useful 256 AES bits key ?

1) Cryptography is no miracle, it's just math. If you use weak password, you get weak protection, and no algorithm saves you from it. This means if you want stable and strong protection, you must use stable and strong key. The one of few options here is to use key generated by OpenSSL itself.

You have to consider user password as worst type of key. Also, many passwords are too plain and dumb: 123, 111, 123456, etc. This is bad for cryptography.

Russian programmer Igor Pavlov who wrote 7zip, has chosen to use compromise solution. He takes user password, calculates SHA-256 function for it (AFAIK U++ has its realization too). Then he adds some calculations/changes to that 256-bit value and the final value is used as a key for AES encryption.

This represents fair protection, which is very much stronger than using user password as key, but at some rate weaker protection than with OpenSSL-generated key. In a number of uses it is rather good and satisfactory protection. Also it allows using protection without storing user password itself which is very good practice. But frankly speaking I haven't heard of SHA output as extremely cryptographically strong combination of bytes. This algorithm has another application field (generating unique digest "far" from original bytes).

2) AFAIK there is no "standard" way to convert user password to key. The best way is to use OpenSSL generated key. You may of course use any function like SHA-256 but you must be aware of the crytpographic strongness/weakness you give to user.

[Updated on: Mon, 22 February 2010 08:43]

Report message to a moderator

Re: Encrypted storage with streaming (OpenSSL, AES) [message #25403 is a reply to message #25402] Mon, 22 February 2010 08:50 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3268
Registered: August 2008
Senior Veteran
Mindtraveller wrote on Mon, 22 February 2010 08:31

koldo wrote on Mon, 22 February 2010 09:54

1) Does it mean that AES cannot be used for saving user files with user defined password ?

However there are programs that include this possibility with AES. For example 7zip offers AES-256 encryption http://www.7-zip.org/7z.html.

2) Is there a standard way to convert a 8 chars user defined password into an useful 256 AES bits key ?

1) Cryptography is no miracle, it's just math. If you use weak password, you get weak protection, and no algorithm saves you from it. This means if you want stable and strong protection, you must use stable and strong key. The one of few options here is to use key generated by OpenSSL itself.

You have to consider user password as worst type of key. Also, many passwords are too plain and dumb: 123, 111, 123456, etc. This is bad for cryptography.

Russian programmer Igor Pavlov who wrote 7zip, has chosen to use compromise solution. He takes user password, calculates SHA-256 function for it (AFAIK U++ has its realization too). Then he adds some calculations/changes to that 256-bit value and the final value is used as a key for AES encryption.

This represents fair protection, which is very much stronger than using user password as key, but at some rate weaker protection than with OpenSSL-generated key. In a number of uses it is rather good and satisfactory protection. Also it allows using protection without storing user password itself which is very good practice. But frankly speaking I haven't heard of SHA output as extremely cryptographically strong combination of bytes. This algorithm has another application field (generating unique digest "far" from original bytes).

2) AFAIK there is no "standard" way to convert user password to key. The best way is to use OpenSSL generated key. You may of course use any function like SHA-256 but you must be aware of the crytpographic strongness/weakness you give to user.


Excellent explanation Smile

I will follow your advice. Anyway, could you add a function to convert an username password into a "fair" protection ?. Thanks Smile

I have checked your demo and now it works well. In a big program where I have applied it, it works well too Smile.

You have done more changes than just a fix Wink. You have removed dependencies to packages Web and Web/SSL.

This afternoon I will upload it to Bazaar. In some hours I will propose a possible application of your useful functions.

Great job !


Best regards
Iñaki
Re: Encrypted storage with streaming (OpenSSL, AES) [message #25410 is a reply to message #23087] Mon, 22 February 2010 11:55 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3268
Registered: August 2008
Senior Veteran
Hello Mindtraveller

I have compiled OpenSSL in MinGW, although only obtaining .dll Sad. It is very easy.

If you want I can include it in T++.


Best regards
Iñaki
Re: Encrypted storage with streaming (OpenSSL, AES) [message #25412 is a reply to message #25410] Mon, 22 February 2010 12:24 Go to previous messageGo to next message
Mindtraveller is currently offline  Mindtraveller
Messages: 917
Registered: August 2007
Location: Russia, Moscow rgn.
Experienced Contributor

Yes, it would be great to add article "Building OpenSSL with MINGW under Windows" and add its reference into tutorial article.
Great work, Koldo, and thanks for checking out my AESStreams!
Re: Encrypted storage with streaming (OpenSSL, AES) [message #25415 is a reply to message #25412] Mon, 22 February 2010 12:46 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3268
Registered: August 2008
Senior Veteran
Hello Mindtraveller

Your package is very useful.

If somebody requires to encrypt seriously a String or raw data from small size to Gb, this is a simple way to do it Smile.

This is not an encryption algorithms catalog. This is just one of the best options with an easy interface.


Best regards
Iñaki
Re: Encrypted storage with streaming (OpenSSL, AES) [message #25420 is a reply to message #25415] Mon, 22 February 2010 15:19 Go to previous messageGo to next message
tojocky is currently offline  tojocky
Messages: 607
Registered: April 2008
Location: UK
Contributor

Koldo,

Can you ad in BAZAAR?

Than you Koldo, and Mindtraveller!

Regards, ion Lupascu (tojocky).

koldo wrote on Mon, 22 February 2010 13:46

Hello Mindtraveller

Your package is very useful.

If somebody requires to encrypt seriously a String or raw data from small size to Gb, this is a simple way to do it Smile.

This is not an encryption algorithms catalog. This is just one of the best options with an easy interface.

Re: Encrypted storage with streaming (OpenSSL, AES) [message #25422 is a reply to message #25420] Mon, 22 February 2010 16:46 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3268
Registered: August 2008
Senior Veteran
tojocky wrote on Mon, 22 February 2010 15:19

Koldo,

Can you ad in BAZAAR?

Than you Koldo, and Mindtraveller!

Regards, ion Lupascu (tojocky).

koldo wrote on Mon, 22 February 2010 13:46

Hello Mindtraveller

Your package is very useful.

If somebody requires to encrypt seriously a String or raw data from small size to Gb, this is a simple way to do it Smile.

This is not an encryption algorithms catalog. This is just one of the best options with an easy interface.




Quote:

This afternoon I will upload it to Bazaar.

Yes. In a few UTC hours Smile


Best regards
Iñaki
Building with MSC9 [message #25617 is a reply to message #25422] Wed, 03 March 2010 23:17 Go to previous messageGo to next message
kohait00 is currently offline  kohait00
Messages: 936
Registered: July 2009
Location: Germany
Experienced Contributor
hi people, great work...

compile with MSC9 worked great, both static and dynamic.
2 quick corrections / hints, though

1) using MSC9 stuff the command is
C:\Temp\openssl-0.9.8m> %comspec% /k ""c:\Program Files\Microsoft Visual Studio 9.0\VC\bin\vcvars32.bat"" x86


2) the out32/ folder offset for the *.lib stuff should be removed in package organizer of AESStreamTest, since the build method setup for LIB stuff already points to "C:\Temp\openssl-0.9.8m\out32"
Re: Building with MSC9 [message #25648 is a reply to message #25617] Sat, 06 March 2010 01:49 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3268
Registered: August 2008
Senior Veteran
Hello Kohait00

Quote:

C:\Temp\openssl-0.9.8m> %comspec% /k ""c:\Program Files\Microsoft Visual Studio 9.0\VC\bin\vcvars32.bat"" x86

Included

Quote:

the out32/ folder offset for the *.lib stuff should be removed in package organizer of AESStreamTest, since the build method setup for LIB stuff already points to "C:\Temp\openssl-0.9.8m\out32"


Sorry, I do not understand.


Best regards
Iñaki
Re: Building with MSC9 [message #25668 is a reply to message #25648] Sun, 07 March 2010 11:24 Go to previous messageGo to next message
kohait00 is currently offline  kohait00
Messages: 936
Registered: July 2009
Location: Germany
Experienced Contributor
hi koldo,

i meant the following:

according to the instructions provided, the LIB directories are to be set to "C:\Temp\openssl-0.9.8m\out32", but in package organizer, the AESStream package includes "out32/libeay32.lib" or "out32dll/libeay32.lib" depending on flags.. this wont work, the libs wont be found.

the LIB directories entry should be set to "C:\Temp\openssl-0.9.8m", thats all.. sorry for misleading post.
Re: Building with MSC9 [message #25674 is a reply to message #25668] Sun, 07 March 2010 13:47 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3268
Registered: August 2008
Senior Veteran
kohait00 wrote on Sun, 07 March 2010 11:24

hi koldo,

i meant the following:

according to the instructions provided, the LIB directories are to be set to "C:\Temp\openssl-0.9.8m\out32", but in package organizer, the AESStream package includes "out32/libeay32.lib" or "out32dll/libeay32.lib" depending on flags.. this wont work, the libs wont be found.

the LIB directories entry should be set to "C:\Temp\openssl-0.9.8m", thats all.. sorry for misleading post.

Hello Kohait00

Now in AESStream doc it appears in the setup the next description:

Quote:

...

2.5. The result is in next folders:
inc32: Include files
out32: *.lib files for static linking
out32dll: *.lib & *.dll files for dynamic linking

...

2.6. Add in "Setup/Build methods/Lib directories" menu, the directory where out32 and out32dll have been copied.


Is it ok ?

And sorry. Your post was not misleading. Just hard to understand for me Razz


Best regards
Iñaki
Re: Building with MSC9 [message #25677 is a reply to message #25674] Sun, 07 March 2010 16:08 Go to previous messageGo to next message
kohait00 is currently offline  kohait00
Messages: 936
Registered: July 2009
Location: Germany
Experienced Contributor
thnak you, no problem.
Re: Encrypted storage with streaming (OpenSSL, AES) [message #25758 is a reply to message #23087] Wed, 10 March 2010 17:09 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3268
Registered: August 2008
Senior Veteran
Hello Mindtraveller

Could you add in AESStream the possibility of accepting keys with length different that 32, 16, ..., perhaps using the SHA-256 algorithm as you suggested before ?


Best regards
Iñaki
Re: Encrypted storage with streaming (OpenSSL, AES) [message #25762 is a reply to message #25758] Wed, 10 March 2010 20:54 Go to previous messageGo to next message
kohait00 is currently offline  kohait00
Messages: 936
Registered: July 2009
Location: Germany
Experienced Contributor
hi koldo,

as far as i got the point of mindtraveler, AES and the other symetric algorithms are not to be thought of beeing based on a "password", a user defined and therefore week combination of signs (which would be scanned first in a brute force attack), but on a statistically well distributed *binary* key (128 bit should be made wise Smile. it is hard for a human beeing to generate one. so the computer will take over and provide some random ones(AES key generator). this key should be thought of as a "password", what it of corse isn't. everything else would diminish the stability of the key.
maybe to get over it, think of it as kind a GUID which you generate once for your application (which in real world communication does not apply Smile. dont think of AES as sort of alphanumerical password dependant encryption algorithm, it's indeed, just as mindtraveler mentioned: math. i had the luck to enjoy some lectures cryptology, and it confuses sometimes. but the first thing we learned there was to forget the idea of passwords / human readable strings as security base.
Re: Encrypted storage with streaming (OpenSSL, AES) [message #25763 is a reply to message #25762] Wed, 10 March 2010 21:33 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3268
Registered: August 2008
Senior Veteran
kohait00 wrote on Wed, 10 March 2010 20:54

hi koldo,

as far as i got the point of mindtraveler, AES and the other symetric algorithms are not to be thought of beeing based on a "password", a user defined and therefore week combination of signs (which would be scanned first in a brute force attack), but on a statistically well distributed *binary* key (128 bit should be made wise Smile. it is hard for a human beeing to generate one. so the computer will take over and provide some random ones(AES key generator). this key should be thought of as a "password", what it of corse isn't. everything else would diminish the stability of the key.
maybe to get over it, think of it as kind a GUID which you generate once for your application (which in real world communication does not apply Smile. dont think of AES as sort of alphanumerical password dependant encryption algorithm, it's indeed, just as mindtraveler mentioned: math. i had the luck to enjoy some lectures cryptology, and it confuses sometimes. but the first thing we learned there was to forget the idea of passwords / human readable strings as security base.

Yes yes, all of you are right Smile

However think about for example a file encrypting software to be used by different people. How would you do it ?

Option 1: The software gives the user a 32 bytes random key
Option 2: The user enters a key


Option 1 seems much stronger. However file and hard disk encrypting software seems to choose option 2.


Best regards
Iñaki
Re: Encrypted storage with streaming (OpenSSL, AES) [message #25767 is a reply to message #25763] Wed, 10 March 2010 22:36 Go to previous messageGo to next message
kohait00 is currently offline  kohait00
Messages: 936
Registered: July 2009
Location: Germany
Experienced Contributor
http://www.winzip.com/aes_info.htm
should explain that its not trivial Smile
Re: Encrypted storage with streaming (OpenSSL, AES) [message #25770 is a reply to message #25767] Thu, 11 March 2010 09:12 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3268
Registered: August 2008
Senior Veteran
kohait00 wrote on Wed, 10 March 2010 22:36

http://www.winzip.com/aes_info.htm
should explain that its not trivial Smile

Hello Kohait00

Thank you for the reference. I will use it.

Coming to the issue, look at this:

- If it is open source, I cannot put the key in the code

- If the program creates a key for the user, and he/she is not let to change it, a 32 bytes password seems too hard to use

- If we use a user defined key, we could include in AESStream:

---1. A SHA 256 possibility to convert user password in a 32 bytes key
---2. The means to avoid a brute force attack.
For example, if AES 256 with a weak user key can resist within and acceptable probability, for example, 1000000 random keys, AESStream could let the main program to enter, for example, 1000 keys per day and after that, AESStream would refuse any additional key.


Best regards
Iñaki
Re: Encrypted storage with streaming (OpenSSL, AES) [message #25772 is a reply to message #25770] Thu, 11 March 2010 10:29 Go to previous messageGo to next message
kohait00 is currently offline  kohait00
Messages: 936
Registered: July 2009
Location: Germany
Experienced Contributor
i dont know if i remember it correctly, but there are several technices combined to achieve encryption of data trggered by a user password.

1) the en/de cryption is done using a *fast* (symetrical) algorithm, like AES (they are blockorientated and relatively similar, only differ in their block functions (F functions, or Feistel Function)

2) the key used there, is the key we were speaking about, and is encrypted and stored with the data. as encryption can be used slow but really strong asymetrical (public / private key) algorithms like RSA.

3) the password thing comes into play with things like diffie hellman secure exchage of information with having it travel over the net.

but its quite a while now, and i may mix it up with things like vpn tunneling and handshaking and so on..

but in any way: encrypting decrypting to fit current standards is far from beeing trivial and involves a lot of steps, password is only a small part of it, maybe we should stick to common technologie here (means in openssl)
Re: Encrypted storage with streaming (OpenSSL, AES) [message #25773 is a reply to message #25772] Thu, 11 March 2010 10:53 Go to previous messageGo to next message
Mindtraveller is currently offline  Mindtraveller
Messages: 917
Registered: August 2007
Location: Russia, Moscow rgn.
Experienced Contributor

The truth is that you MUST
1) Use strong key with AES, not password
2) Not to hardcode the key in the code in ANY way

So what is the solution? You take user password. And then you DERIVE strong key from it. Then you "forget" user password, you just don't need it at all. You do encryption with that relatively strong key (i.e. SHA from user password - see my recent comment).

Next time user enters password, you derive the key with the same function (i.e. SHA) and try to decompress AESStream. If decomression fails, then original password and the one entered is not the same (incorrect password).

It is really not that hard.
Re: Encrypted storage with streaming (OpenSSL, AES) [message #25774 is a reply to message #25773] Thu, 11 March 2010 11:01 Go to previous messageGo to previous message
koldo is currently offline  koldo
Messages: 3268
Registered: August 2008
Senior Veteran
Mindtraveller wrote on Thu, 11 March 2010 10:53

The truth is that you MUST
1) Use strong key with AES, not password
2) Not to hardcode the key in the code in ANY way

So what is the solution? You take user password. And then you DERIVE strong key from it. Then you "forget" user password, you just don't need it at all. You do encryption with that relatively strong key (i.e. SHA from user password - see my recent comment).

Next time user enters password, you derive the key with the same function (i.e. SHA) and try to decompress AESStream. If decomression fails, then original password and the one entered is not the same (incorrect password).

It is really not that hard.

Could you implement this in AESStream ?


Best regards
Iñaki
Previous Topic: Protect packages - split code encryption,client and server
Next Topic: Added single and double linked lists
Goto Forum:
  


Current Time: Tue Dec 07 16:14:26 CET 2021

Total time taken to generate the page: 0.01394 seconds