Nonexistant subscribed folders are listed in mail view, but can't be unsubscribed from

[brandond]

Our old webmail system had a bug in its folder renaming code that would leave a subscription entry for the old folder name, in addition to adding a subscription for the new folder. Dovecot does not validate that the subscription list contains only valid entries, and returns these nonexistent folders to the client.

Roundcube's handling of this is somewhat odd, in that it shows the nonexistant folder (as empty) in the mail interface, but not in the folder subscription list. This leaves you with a 'ghost' folder that you can't unsubscribe from, short of going to another client and doing it there.

Roundcube should either hide the folder in the Mail view, or display it in the folder list (perhaps with some indication that it doesn't exist) so it can be unsubscribed from.

[alec]

#1485988 marked as duplicate.

[serpico7456]

Mailbox view showing non-existent folder

[serpico7456]

imap subscription view which doesn't show the same folder as the mailbox view

[serpico7456]

The problem of leaving subscriptions for non-existent folders is also caused by other mail clients, as well as if the folder is manually deleted on the mail server. While I agree Dovecot should validate subscription lists, this doesn't change the fact the Roundcube's handling of this is inconsistent. As brandond mentions, RC should either hide the folder in the mailbox view (as is already does in the subscription view) OR show it in the subscription view (as it already does in the mailbox view)--It should not show it in some contexts and not in others.

Our temporary work around has been to disable IMAP folder subscriptions via the bundled 'subscriptions_option' plugin. This way non-existent folders will not appear in the mailbox view.

See the attached screen shots which demonstrate the problem.

[alec]

Actually IMAP RFC doesn't require to validate subscription list on server-side, so maybe we should do this in Roundcube. We need to call LSUB for subscriptions list and LIST for existance check. Currently we aren't do this for performance reasons.

[alec]

...or we could use LIST (SUBSCRIBED) instead of LSUB on servers with LIST-EXTENDED (RFC5258) support.

[alec]

With following patch "ghost" folders are removed when user enters Folders tab. I'm not sure if this is proper fix for this issue.

--- manage_folders.inc  (wersja 3870)
+++ manage_folders.inc  (kopia robocza)
@@ -255,6 +255,12 @@
     $RCMAIL->user->save_prefs(array('message_threading' => $a_threaded));
   }

+  $a_ghosts = array_diff($a_subscribed, $a_unsubscribed);
+  unset($a_unsubscribed);
+  // Remove subscribed but non-existing folders (#1486225)
+  foreach ($a_ghosts as $ghost)
+    $IMAP->unsubscribe($ghost);
+
   $checkbox_subscribe = new html_checkbox(array(
     'name' => '_subscribed[]',
     'onclick' => JS_OBJECT_NAME.".command(this.checked?'subscribe':'unsubscribe',this.value)",

[brandond]

Patch looks good to me. Honestly I'd be happy to see Roundcube go as far as sending an unsubscribe if they don't exist.. but just hiding them is good too, and probably safer. Any chance we could see this committed?

[brandond]

Actually I'm an idiot, that's exactly what it does ;) Sorry for the noise. I'd still be happy to see it committed, unless you'd prefer to take the LIST (SUBSCRIBED) approach as a safer alternative.

[alec]

IMAP RFC says:

      A server MAY validate the mailbox argument to SUBSCRIBE to verify
      that it exists.  However, it MUST NOT unilaterally remove an
      existing mailbox name from the subscription list even if a mailbox
      by that name no longer exists.

           Note: This requirement is because a server site can
           choose to routinely remove a mailbox with a well-known
           name (e.g., "system-alerts") after its contents expire,
           with the intention of recreating it when new contents
           are appropriate.

So, now I think we shouldn't unsubscribe non-existing folders, but we should display them in Settings/Folders? with option to unsubscribe.

[brandond]

That'd be good too. Think you'd be able to put together a patch for that anytime soon? I was just about to apply the unsubscribe patch to my installation, but I'd be glad to do the 'right thing' instead if a patch was available.

[alec]

Fixed in [3870bec7], so non-existent folders are not listed.

[dziobak]

Hi Alec, after this change I can't see my public and shared mailboxes and I can't subscribe them back.

[dziobak]

look post before.

[alec]

Show imap_debug. I feel they are marked as \Noselect.

[dziobak]

unfortunetly:

[27-Oct-2010 09:43:07 +0200]: S: * LIST (\Subscribed \NonExistent) "/" "Public/AddressBook"
[27-Oct-2010 09:43:07 +0200]: S: * LIST (\Subscribed \NonExistent) "/" "Public/PublicMails"
[27-Oct-2010 09:43:07 +0200]: S: * LIST (\Subscribed \NonExistent) "/" "Public/"

[alec]

If they're not exists, why you care of them?

[brandond]

If you're using Dovecot, this could be an issue with how you are storing subscriptions for your public namespace. See:

​http://www.dovecot.org/list/dovecot/2010-October/054310.html

Basically, LSUB or LIST (SUBSCRIBED) will return odd results if you're storing subscription entries for one namespace in another namespaces' subscription file.

If course it could be something else entirely, but this is something I ran into recently.

[dziobak]

because the public mailboxes are working well. this is a special configuration ;-)
the same is with shared mailboxes:

[27-Oct-2010 09:51:59 +0200]: S: * LIST (\Subscribed \NonExistent) "/" "Shared/dziobak/root"
[27-Oct-2010 09:51:59 +0200]: S: * LIST (\Subscribed \NonExistent) "/" "Shared/tester2/sdfgsg"
[27-Oct-2010 09:51:59 +0200]: S: * LIST (\Subscribed \NonExistent) "/" "Shared/tester2/sdfgsg/gsdfgf"
[27-Oct-2010 09:51:59 +0200]: S: * LIST (\Subscribed \NonExistent) "/" "Shared/tester2/sdfgsg/gsdfgf/vtyjyfgj"
[27-Oct-2010 09:51:59 +0200]: S: * LIST (\Subscribed \NonExistent) "/" "Shared/tester2/sdfgsg/gsdfgf/vtyjyfgj/rtdhrthxs"
[27-Oct-2010 09:51:59 +0200]: S: * LIST (\Subscribed \NonExistent) "/" "Shared/tester2/sdfgsg/gsdfgf/vtyjyfgj/rtdhrthxs/zhfhg"
[27-Oct-2010 09:51:59 +0200]: S: * LIST (\Subscribed \NonExistent) "/" "Shared/"

and it is working well too.

[alec]

So, are these folders exist? What returns LIST "" "*" command? Please, attach complete imap_debug log.

[dziobak]

I can't enter those folders from RC, because I don't see them now and I don't want to make revert. I can give you the telnet output.

. list "" "*"
* LIST (\HasNoChildren) "/" "Trash"
* LIST (\HasNoChildren) "/" "Junk"
* LIST (\HasNoChildren) "/" "Informacje"
* LIST (\HasNoChildren) "/" "Sent"
* LIST (\HasNoChildren) "/" "Drafts"
* LIST (\HasNoChildren) "/" "Archives"
* LIST (\HasNoChildren) "/" "Kalendarz"
* LIST (\HasNoChildren) "/" "Wiadomo&AVs-ci-&AVs-mieci"
* LIST (\HasNoChildren) "/" "INBOX"
* LIST (\HasNoChildren) "/" "Public/AddressBook"
* LIST (\HasNoChildren) "/" "Public/PublicMails"
* LIST (\Noselect \HasChildren) "/" "Shared/dziobak"
* LIST (\Noselect \HasChildren) "/" "Shared/tester2"
* LIST (\HasNoChildren) "/" "Shared/dziobak/root"
* LIST (\HasChildren) "/" "Shared/tester2/sdfgsg"
* LIST (\HasChildren) "/" "Shared/tester2/sdfgsg/gsdfgf"
* LIST (\HasChildren) "/" "Shared/tester2/sdfgsg/gsdfgf/vtyjyfgj"
* LIST (\HasChildren) "/" "Shared/tester2/sdfgsg/gsdfgf/vtyjyfgj/rtdhrthxs"
* LIST (\HasNoChildren) "/" "Shared/tester2/sdfgsg/gsdfgf/vtyjyfgj/rtdhrthxs/zhfhg"
. OK List completed.
. select Shared/dziobak/root
* OK [CLOSED] Previous mailbox closed.
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk Junk)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk Junk)] Flags permitted.
* 716 EXISTS
* 0 RECENT
* OK [UNSEEN 716] First unseen.
* OK [UIDVALIDITY 1269599970] UIDs valid
* OK [UIDNEXT 4199] Predicted next UID
* OK [HIGHESTMODSEQ 175] Highest
. OK [READ-WRITE] Select completed.

[alec]

So, they are existing. It looks like dovecot bug. Maybe you could confirm this on dovecot mailing list.

[dziobak]

I have version 1.2.14. so Timo won't correct this. I use only ports for FreeBSD without patches. You can make a config option for compatibility with this bug.

[brandond]

It might not be a bug - you may just have to change your Dovecot configuration to store subscriptions in the proper namespace. You should at least ask.

[alec]

If this is a bug I'll provide force_lsub option, but I need confirmation from Timo.

[dziobak]

Here is Timo confirmation:
​http://www.dovecot.org/list/dovecot/2010-October/054372.html

[alec]

Added imap_force_lsub option in [f75f65ce].

[jonesy16]

I can confirm that the same Dovecot issues exists when configuring shared namespaces in the shipping version of the IMAP server on OS X Server 10.6+. The imap_force_lsub option worked for me.

[smithmb]

Just to add, we saw this same issue at UF with various mail clients. The support work to undo it was a nightmare, and this patch helped a ton :)

[frecon]

I am still, in 0.5.2, not able to unsubscribe from non-existing subscribed folders. Is this supposed to be fixed with the imap_force_lsub option or was that solution of the problem disregarded? I'm running courier-imap.

[alec]

Enable imap_debug and provide the log. What you mean you are not able? Any error or not possible to unselect a checkbox?

[frecon]

Replying to alec:

Enable imap_debug and provide the log. What you mean you are not able? Any error or not possible to unselect a checkbox?

As described in the initial ticket description, the non-existing subscribed folders are displayed in the E-Mail - Folders list but not in the Settings - Folders list. I can not open the folders and I cannot unsubscribe to them using roundcube.

The IMAP log (imap), the E-Mail Folders list (e-mail-folders-list.jpg) and the Settings Folders list (settings-folders-list.jpg) are attached.

[mtu]

I can confirm this fixed the problem for me. My IMAP daemon is dovecot 1.1.3. Thanks a lot!