Attachment Excessive Memory Use Error

[JohnDoh]

Hi,

I know tickets have been created about this before but I cant find the exact one and many of them seem to be lost in some kind of "dupicate of" hell. I thought it was probably easier to just start a new one. I applogies if I am repeating others informaiton but I cant find the previous tickets.

The amount of memory required to send an email with attachments seems to massivly out way the size of the attachments giving an error like:

"Fatal error: Allowed memory size of blah bytes exhausted (tried to allocate blah bytes)"

in the error log.

(thrown by the quotedata() function in program/lib/Net/SMTP.php)

Examples: required more than 64mb to send 7mb attachment or 25mb to send 5.5

More people are now reporting this on the forum (http://roundcubeforum.net/forum/index.php?topic=1811.0)

I know that the attachment size limits (which I think only apply to individual files, not the combined size) and the php memory limits can be altered but i dont think this counts as a solution when the difference in requirements is so great

This still occurs in SVN890

Thanks and sorry again if I am repeating stuff but I cant track down the previos tickets which I know exist about this exact issue.

[thomasb]

Well the message body is passed as a string and is altered by Net/SMTP.php. This means that PHP copies the whole string in memory. The only solution is to re-write the PEAR classes to work with streams instead of string variables. Any volunteers out there?

[zdali]

I'll foolishly put my hand up for this. I already have a working prototype for this.

[seansan]

Marked as later until we have a solution

[till]

My settings:

memory_limit = 256M
max_execution_time = -1

IMO not a RoundCube issue, but more subject to fine-tuning your configuration. My setting is not too high either (at least, IMHO) since the current memory_limit setting PHP is shipped with is 128M(B).

[oleole]

This is a major issue for shared hosting, VPS hosting and other types of hosting servers.

The solution to raise the memory limit to 256MB is unrealistic and just plainly wrong. Hosting servers are very limited in memory and 256MB could be the entire memory of a whole VPS server.

Disabling the maximum execution time is just plain stupid. That option is there to avoid lockups, deadlocks and on a production server is a must-have. If you disable it, you risk of bringing the whole server down with a few astray infinite loops.

This is definitely an issue, since roundcube is getting ready to be included in major hosting control panels like cPanel. Shooting our selfs in the foot with this will limit roundcube's user base.

[kaouete]

Hello,

I hope this problem will not be fixed like till proposed.

By looking at the forum post the bug is refering to, it is obvious there is a problem in the code ...

Thanks :)

[till]

I installed a fresh PHP5 on my sandbox over the weekend, the current value is already 128 MB. Can anyone try with this value?

Not to be rude here, but if 256 MB is unrealistic for you guys, then what exactly is realistic here? 16 MB? 32 MB?

I am not closing this issue, however, I am postponing it to 0.1.1 for review.

[jbarbuc]

I have a patch for this issue that uses file streams as suggested above.

With this patch I was able to send a 10MB sized file with a 5MB memory limit, but I have no plans on keeping the limit so low.

See http://public.lanl.gov/jbarbuc/patches/roundcube/ for information and http://public.lanl.gov/jbarbuc/patches/roundcube/roundcubemail-0.1.attach.patch for the patch.

[till]

Traced this problem with a bit of help from the Mail_Mime maintainer, I think we should "free" the $MAIL_MIME contents ASAP, before we hand them off to smtp_mail (which wraps Net_SMTP). This would free a lot of memory.

For reference: http://trac.roundcube.net/browser/trunk/roundcubemail/program/steps/mail/func.inc#L1388

[till]

But (of course) we can't delete it there, because after rcmail_deliver_message, it's also used to build the message for "sent".

Putting this on 0.1.2. Also had no chance to check this "streaming" patch yet.

[till]

We can't apply the patch because you heavily patch external libs to use RoundCube function calls, etc..

Let me know if you would like to work on this, email me - till'at'roundcube'dot'net.