Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » Developing U++ » U++ Developers corner » Pop3 class and reference examples for U++
Pop3 class and reference examples for U++ [message #41753] Fri, 17 January 2014 01:49 Go to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello,

I'ts been a long time since I upload some code to bazaar and I would like to upload my brand new Pop3 class (and reference examples) to the bazaar if the admins (Koldo?) could grant me the SVN/bazaar access.

Pop3 is a Upp::TcpSocket derived POP3 encapsulation class, very much like the Upp::Smtp class in its public api.
While it conforms to the RFC 1939, it is implemented with simplicity in mind. Currently, it is in beta status. While its public api is stable, internals will probably change, for I am planning to add Capabilities (CAPA command) support. So there is of couse room for improvements, such as writing a much flexible and Upp/C++ friendly parser to substitute sscanf command, fixing bugs, refactoring the code, etc.

I tried to test it as extensively as I can, with different and popular POP3 providers.

And I wrote a simple console based reference example (again, similar to the SMTP reference example), which can retrieve message headers. I am also writing a GUI based one. I will upload it here in the next days.

Pop3 class requires OpenSSL. If you are using Linux, you probably have it. If you are Using windows, you can either get the source and compile it, or better, simply download the precompiled binaries from: (http://www.openssl.org/related/binaries.html) and configure it using TheIDE's "Setup"->"Build Methods..." menu.

Pop3 class and relevant example code are tested under:

U++ version 6738

Arch Linux: Linux 3.12 i686 Kernel with KDE SC 4.12.
Windows XP SP3 (i686)



Any suggestions, bug reports are always welcome.

And a happy new year to everyone!

P.s.: There are two helper functions in Pop3Example reference code: One ise DecodeEncodedString(): to decode QuotedPrintable/Q encoded texts. And the other is FindHeadersElement(): to get header sections. I wrote them in a hurry and long time ago so they are not meant to stay. I am planning to write an InternetMessage/EMail class, so they will be replaced with proper and fresh methods.


Regards.



[Updated on: Fri, 17 January 2014 01:57]

Report message to a moderator

Re: Pop3 class and reference examples for U++ [message #41755 is a reply to message #41753] Fri, 17 January 2014 08:12 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3355
Registered: August 2008
Senior Veteran
Hello Oblivion

Please check this link about OpenSSL installing if it is good for you.


Best regards
IƱaki
Re: Pop3 class and reference examples for U++ [message #41756 is a reply to message #41755] Fri, 17 January 2014 08:27 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Looks good, except

sscanf(line, "%s %s", &tag1[0], &tag2[0]);

lines... Smile (using sscanf with fixed buffers is security problem, also instead of &tag1[0] you can write ~tag1).

With your permission, I would do CR to it, fix some hard edges and put into uppsrc as plugin/POP3.
Re: Pop3 class and reference examples for U++ [message #41758 is a reply to message #41756] Fri, 17 January 2014 12:00 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Quote:


Looks good, except

sscanf(line, "%s %s", &tag1[0], &tag2[0]);

lines... (using sscanf with fixed buffers is security problem, also instead of &tag1[0] you can write ~tag1).

With your permission, I would do CR to it, fix some hard edges and put into uppsrc as plugin/POP3.



Hello mirek.

Formally: permission granted. Informally: Of course, please feel free correct it, fix it and put it into uppsrc. I would be happy to contribute to U++ anyways Smile Also I am working on a Ftp class, so any corrections on or fine-tuning of this code will make the Ftp class and my understanding of the TcpSocket class better.

By the way, I have two questions:

1. Do you allow patches? I was planning to add CAPA command and replace the sscanf with a flexible parser. If you are going to add these, fine. If not, I would like to work on it.

2. This is somewhat a general question. Do you have any future plans to add a Scan() command (like the Format() command) to U++?


Quote:


Hello Oblivion

Please check this link about OpenSSL installing if it is good for you.



Thanks for the information, koldo!

Regards.


[Updated on: Fri, 17 January 2014 12:07]

Report message to a moderator

Re: Pop3 class and reference examples for U++ [message #42570 is a reply to message #41758] Tue, 25 March 2014 10:07 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
I have done some minor changes and added class InetMessage to simplify parsing. Otherwise, it is now Core/POP3, reference/Pop3. Thanks a lot, this is very helpful.

Quote:


2. This is somewhat a general question. Do you have any future plans to add a Scan() command (like the Format() command) to U++?



Usually I prefer CParser. Anyway, your use of sscanf inspired me to introduce SplitTo function....

Mirek
Re: Pop3 class and reference examples for U++ [message #42605 is a reply to message #42570] Wed, 26 March 2014 10:34 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
I believe that your original code for multiline responses is wrong (found that hard way on actual message).

I have changed the code to

				for(;;) {
					line = GetDataLine();
					if(line.IsEmpty())
						break;
					if(line == ".\r\n") {
						LLOG("<< ...");
						return true;
					}
					if(*line == '.')
						data << line.Mid(1);
					else
						data << line;
				}


Please check...

Mirek
Re: Pop3 class and reference examples for U++ [message #42610 is a reply to message #42605] Wed, 26 March 2014 12:37 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello Mirek,

Ah.. I didn't encounter any actual errors but yes, looking more closely, you are right that was my fault, sorry. And thanks for correcting it. Smile

Other than that, after furter examining the code, I shall report an error in the Pop3::GetListItems() method:
Current code omits the last line of the list. So I corrected it.


bool Pop3::GetListItems(ValueMap& list, dword type1, dword type2)
{
	StringStream s(data);
        for(;;) {
		String line = s.GetLine();
		if(s.IsError())
			return false;
		if(s.IsEof())                                // << ---- Omits last line of the list. EOF checking should be moved to top.
			break;
		Vector<String> s = Split(line, ' ');
		if(s.GetCount() < 2)
			return false;
		list.Add(Scan(type1, s[0]), Scan(type2, s[1]));
	}
	return true;
}


Corrected version:

bool Pop3::GetListItems(ValueMap& list, dword type1, dword type2)
{
	StringStream s(data);
	while(!s.IsEof()) {
		String line = s.GetLine();
		if(s.IsError())
			return false;
		Vector<String> s = Split(line, ' ');
		if(s.GetCount() < 2)
			return false;
		list.Add(Scan(type1, s[0]), Scan(type2, s[1]));
	}
	return true;
}


Other than that, everything seems fine.


P.S. Thanks a lot for the SplitTo() function, it is and will be very helpful. Too bad those handy utility functions got undocumented.
When I have time I'll examine and document them.

Regards.


[Updated on: Wed, 26 March 2014 18:29]

Report message to a moderator

Re: Pop3 class and reference examples for U++ [message #42611 is a reply to message #41753] Wed, 26 March 2014 18:31 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Also I propose a change:
I was looking into the possibility of using TcpSocket::GetLine() instead of Pop3::GetDataLine() method and come up with the below modification.
You are the expert, so please comment if this is valid. (It seems so and works, but again, I am suspicious).


bool Pop3::PutGet(const String& s, bool multiline, bool nolog)
{

        // Put() request.
        if(!s.IsEmpty()) {
         	    if(!nolog)
                        LLOG(">> " << TrimRight(s));
                if(!PutAll(s)) {
                        LLOG("-- " << GetLastError());
                        return false;
                }              
        }
        // Get() response.
        data.Clear();
        String line = GetLine();
        if(!line.IsVoid()) {
                LLOG("<< " << line);
                if(line.StartsWith("+OK")) {
                        if(!multiline) {
                                data.Cat(line);
                                return true;
                        }
                        else
                                for(;;) {
                                        line = GetLine();
                                        if(line.IsVoid()) // IsEmpty() cannot be used here.
                                                break;
                                        if(line == ".") {
                                                LLOG("<< ...");
                                                return true;
                                        }
                                        data.Cat(*line == '.' ? line.Mid(1) : line);
                                        data.Cat("\r\n");
                                }
                }
                else
                if(line.StartsWith("-ERR"))
                        error = line;
        }
        LLOG("-- " << GetLastError());
        return false;

}



Basically, it appends the CRLF after downloading. This way, we can discard GetDataLine() method in favor of native TcpSocket method.

Regards.


[Updated on: Wed, 26 March 2014 18:35]

Report message to a moderator

Re: Pop3 class and reference examples for U++ [message #42655 is a reply to message #42611] Thu, 27 March 2014 08:45 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Yeah, that is better.
Re: Pop3 class and reference examples for U++ [message #42661 is a reply to message #42655] Thu, 27 March 2014 15:24 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello Mirek,

Latest revision of Pop3 has a constant value (int LINE_MAX) used in GetLine(). But afaik, LINE_MAX is actually a posix macro used by gcc. So it gives an error:

../Ultimate++/uppsrc/Core/POP3/POP3.cpp: In member function 'bool Pop3::PutGet(const Upp::String&, bool, bool)':
..Ultimate++/uppsrc/Core/POP3/POP3.cpp:114:12: error: expected unqualified-id before numeric constant
  const int LINE_MAX = 20000000;
            ^


Regards.


Re: Pop3 class and reference examples for U++ [message #42662 is a reply to message #42661] Thu, 27 March 2014 15:38 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Fixed. I have also hugely refactored InetMessage....
Re: Pop3 class and reference examples for U++ [message #42697 is a reply to message #42662] Sun, 30 March 2014 01:48 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello Mirek,

I refactored the DecodeHeaderValue() helper function of InetMessage.
The old code was written hastily and supplied for demonstration purpose only; it had some problems, so it shouldn't be used.
I attached the refactored version (tested with my several mailboxes), I would be grateful if you could review it. Smile

Regards.


[Updated on: Sun, 30 March 2014 04:00]

Report message to a moderator

Re: Pop3 class and reference examples for U++ [message #42698 is a reply to message #42697] Sun, 30 March 2014 09:29 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Thanks, I was thinking about this too.

Mirek
Re: Pop3 class and reference examples for U++ [message #42848 is a reply to message #42698] Thu, 10 April 2014 01:24 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello Mirek,

I've noticed that you refactored the INetMessage and SMTP classes. When I looked into the code of Smtp class I noticed that the quoted-printable encoding was hard coded into the class.
So I took the opportunity to improvise on the existing codebase. While I did not touch the Smtp class, but I wrote a generic QPEncode() function to accompany the existing QPDecode() function.
There seem to be two problems with the current QP encoding of Smtp (if I am not missing anything, or got something totally wrong Smile ):

1) It violates the maxlen (should be <= 76 chars) of quoted-printable text (RFC 1521, rule 5).
2) It does not encode horizontal tab (=09) or space (=20) at the EOL, just before the CRLF (it should). (This statement is wrong, it turns out that I misread the code, it was only implicit. Embarassed )

So, I created the function and patched accordingly the source files (inet.h, inet.cpp) and the documentation (inet$en-us.tpp)

I tested it with a lot of strings, but as usual, it always needs to be tested more. Please find enclosed the patched files.
As usual, I would be grateful if you could please review it.

EDIT: Fixed misinformation.

Regards.


[Updated on: Thu, 10 April 2014 12:35]

Report message to a moderator

Re: Pop3 class and reference examples for U++ [message #42852 is a reply to message #42698] Thu, 10 April 2014 16:14 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello Mirek,
I updated the QPEncode() function using the original code snippet in SMTP. It turned out that the coversion of HT and SP was implicitly present, I guess I should read more carefully next time.
Anyways, my previous version of QPEncode was working but this version is more compact (unnecessary code removed) and also encodes the last space character right before the EOF to QP, if no CRLF present at the end.

Regards.
  • Attachment: InetUtil.cpp
    (Size: 13.95KB, Downloaded 365 times)


Re: Pop3 class and reference examples for U++ [message #42948 is a reply to message #42698] Wed, 16 April 2014 03:00 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello Mirek,
I see that you recently added ParseMessageIDs() method to the InetMessage class.
The code block in that function is something I encounter often. Especially when parsing some text (you can even see it in the Pop3::GetTimestamp()).
Maybe we should put that code under String utilities and change the code accordingly?
So I wrote the following Slice() functions which you can also come by in some other C++/Java frameworks.


int Slice(const String& s, String& d, const String& delim1, const String& delim2, int pos = 0)
{
	int b = -1, e = -1;
	if((b = s.Find(delim1, pos)) == -1 || (e = s.Find(delim2, b += delim1.GetLength())) == -1) 
		return -1;
	d = s.Mid(b, e - b);
	return e += delim2.GetLength();
}

int Slice(const WString& s, WString& d, const WString& delim1, const WString& delim2, int pos = 0)
{
	int b = -1, e = -1;
	if((b = s.Find(delim1, pos)) == -1 || (e = s.Find(delim2, b += delim1.GetLength())) == -1) 
		return -1;
	d = s.Mid(b, e - b);
	return e += delim2.GetLength();
}

int Slice(const String& s, String& d, int delim1, int delim2, int pos = 0)
{
	int b = -1, e = -1;
	if((b = s.Find(delim1, pos)) == -1 || (e = s.Find(delim2, ++b)) == -1) 
		return -1;
	d = s.Mid(b, e - b);
	return ++e;
}

int Slice(const WString& s, WString& d, int delim1, int delim2, int pos = 0)
{
	int b = -1, e = -1;
	if((b = s.Find(delim1, pos)) == -1 || (e = s.Find(delim2, ++b)) == -1) 
		return -1;
	d = s.Mid(b, e - b);
	return ++e;
}

String Slice(const String& s, const String& delim1, const String& delim2)
{	
	String r;
	return (Slice(s, r, delim1, delim2) == -1) ? String::GetVoid() : r;
}

WString Slice(const WString& s, const WString& delim1, const WString& delim2)
{	
	WString r;
	return (Slice(s, r, delim1, delim2) == -1) ? WString::GetVoid() : r;
}

String Slice(const String& s, int delim1, int delim2)
{	
	String r;
	return (Slice(s, r, delim1, delim2) == -1) ? String::GetVoid() : r;
}

WString Slice(const WString& s, int delim1, int delim2)
{
	WString r;
	return (Slice(s, r, delim1, delim2) == -1) ? WString::GetVoid() : r;

}


They are really useful for parsing text. I already patched my local copy of upp (with its api reference doc in "String utility functions" section).
By the way, I can upload the patched files if you have time to review them and also approve them.

Regards.


[Updated on: Wed, 16 April 2014 14:28]

Report message to a moderator

Re: Pop3 class and reference examples for U++ [message #43061 is a reply to message #42852] Mon, 28 April 2014 17:48 Go to previous message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Oblivion wrote on Thu, 10 April 2014 16:14
Hello Mirek,
I updated the QPEncode() function using the original code snippet in SMTP. It turned out that the coversion of HT and SP was implicitly present, I guess I should read more carefully next time.
Anyways, my previous version of QPEncode was working but this version is more compact (unnecessary code removed) and also encodes the last space character right before the EOF to QP, if no CRLF present at the end.

Regards.


Thanks, applied.

Mirek
Previous Topic: Value size ?
Next Topic: Skylark: Should escape character '$' be replaced to avoid name clash with Jquery?
Goto Forum:
  


Current Time: Thu Mar 28 22:53:53 CET 2024

Total time taken to generate the page: 0.01228 seconds