Discussion with peppiv

At the end of this article.
-- Hannes

Hi; sorry, I'm new here, and I'm interested in yor patches/fixes. But I don't see any attachments. Where do I find them?
thanks, John

Hannes,

I'm using Zope 2.7.7, Plone 2.1-rc2, Python 2.3.5, GRUF 3.3 (with your patch) and RUF (with your update).

Seems a bit cleaner and faster. Authenticates me right away, creates a User but not Member. I still have to go back and give rights to the User so that they can edit pages. Even when I assign rights to edit preferences, it doesn't allow it. I've tried to create a user or member before I log in and the same applies. It's like everything works but that last step.

I'm trying it right now and will let you know how it works.

Super Thanks

... that you need to delete all users in your RemoteUserFolder instance. If that doesn't work, try removing acl_users completely and creating it from scratch as a GRUF with a RUF as a source. I think the changes I made to RUF are incompatible with existing RUFUser instances that are pickled in ZODB.

Thank you for posting your work. It has been extremely helpful and educational.

When I tried the new RUF and GRUF3.3 patch, I got this error -
Traceback (innermost last):
Module ZPublisher.Publish, line 119, in publish
Module Zope2.App.startup, line 215, in commit
Module transaction._manager, line 84, in commit
Module transaction._transaction, line 382, in commit
Module transaction._transaction, line 380, in commit
Module transaction._transaction, line 425, in _commitResources
Module ZODB.Connection, line 462, in commit
Module ZODB.Connection, line 503, in _commit
Module ZODB.Connection, line 525, in _store_objects
Module ZODB.serialize, line 330, in serialize
TypeError: Can't pickle objects in acquisition wrappers.

When I reinstalled the old version of RUF, it creates the user with the same permissioning problems. Then I switched back to the new RUF and it worked like the old. Once the user was there, it didn't have a problem with the acquisition wrapper.

Seems to be a problem with
(import pickle)
def __getstate__(self):
getstate = w.__getstate__
getstate()
Traceback (most recent call last):

...

TypeError: Can't pickle objects in acquisition wrappers.

I couldn't tell from the new RUF whether it was implicit or explicit acquisition. Sorry - weak python/zope skills.
I'm using
Plone version: 2.1-rc1
Zope version: (Zope 2.8.0-final, python 2.3.5, linux2)

Although Plone 2.1-rc2 just came out today.

As announced in my previous comment, I attached to this article the GRUF patch for GRUF 3.3 and a heavily modified version of RemoteUserFolder.

Not sure about Z3. They talked about it at the Plone Symposium in New Orleans, but I don't remember the discussion.

Been unraveling the whole thing too. It's definitely a roles thing. Still not sure why it creates a User, but not a Member. Might be a Plone 2.1 thing.

I heavily modified RUF today and got the "domain roles" thing to work. All in all, I think it's a nasty hack but who cares?

Programming for Zope would be so sweet if it wasn't for that stupid acquisition thingy. It's a can of worms just like multiple inheritance and it's Zope's number one cause of bugs. I'm so glad they tried to get it under control in Z3. Do you know what the plans are for migrating Plone to Z3?

Anyway, I'm not at the office right now but I'll send you the modified RUF on monday if you want to give it a shot.

If RemoteUserFolder is that far gone, I completely understand if you don't want to put any more time into it.

I don't think there's any real advantage to the GRUF 3.3. We're seeing it more from the Plone 2.1 side. Lots of great enhancements such as diminishing privileges within a folder structure. Things we used to try to control with ugly workarounds using workflows.

Yes we did use the modified version of mod_ntlm. I went back and checked because I was suspect of that too.

I'm going to keep plugging at it. Maybe make some major changes to RUF. I emailed the person who developed it, but never got a response. So much of it works, yet much of it doesn't.

I tried to find out why the "domain roles" feature doesn't work. It turns out that RemoteUserFolder is more or less broken. Half of the code (the entire RemoteUser class) dead, i.e. never used because there are two instances of _doAddUser() of which only the second one is actually used. This second _doAddUser() does not instantiate RemoteUser objects but standard User objects. Hence the "domain roles" code is never invoked.

If I remove the second _doAddUser instance, things get even worse: GRUF stops working entirely. I think it's got to do with Zope's aquisition and persistance. I don't know how much more time I want to put into this. I'll keep you up to date.

BTW: I upgraded to GRUF 3.3 too but I don't really see the advantage of it. :-o

I think your problem is rooted in Apache or the domain controller. At least that's how I interpret your apache log snippet. Are you using Michael Cai's modified version of mod_ntlm or the original one?

I'm not sure if this is realated to your problem but I tried to find out why the "domain roles" feature doesn't work. It turns out that RemoteUserFolder is more or less broken. Half of the code (the entire RemoteUser class) dead, i.e. never used because there are two instances of _doAddUser() of which only the second one is actually used. This second _doAddUser() does not instantiate RemoteUser objects but standard User objects. Hence the "domain roles" code is never invoked.

If I remove the second _doAddUser instance, things get even worse: GRUF stops working entirely. I think it's got to do with Zope's aquisition and persistance. I don't know how much more time I want to put into this. I'll keep you up to date.

BTW: I upgraded to GRUF 3.3 too but I don't really see the advantage of it. :-o

Sorry Hannes. We've had to take a left turn. We upgraded to Plone 2.1rc. After upgrading Python/Zope/Plone and reinstalling RemoteUserFolder and your patch on GRUF 3.3, I'm back to the same place. The only difference is when I go to Members in my Plone site, there's nothing but index_html there. No member has been created, even though I've been authenticated and successfully logged in (yes, I checked Auto Add Users in RUF).

I did check the two things you suggested and "Set own properties" has Authenticated checked. Acquire was not checked. The user in RUF did have roles highlighted. I added myself to the group Administrators and I do have some capabilities of 'editing' and 'properties', but still no preferences.

I'm digging through RUF and GRUF to see if I can pinpoint why it's not creating a member or member folder. So much of it is working correctly. The SSO is really sweet.

Sorry again for having to change versions. It makes it a little harder to troubleshoot.

Could you go to the Zope management interface and click on your plone site? Click the Security tab. Does the "Set own properties" permission have a checkmark under the Authenticated role? Click on acl_users in your plone site (not the acl_users at the site root), click on sources, click on Remote User Folder. You should see the users in your Remote User Folder. Does the user for which "my preferences" fails have any roles assigned to it? Assigned roles need to be *hilited*, not just listed.

We'll finish this. I enjoy these types of challenges.

I tried as you requested, to assign the necessary roles to the domain user (as Zope mgr). But still could not get it to work. You're analysis seems correct. Not sure why I couldn't get it to work right.

I tried changing every other security permission and assigned role hoping to overcome the basic auth. But still the same result. I installed 'Verbose Security' but got no deeper message. I did find something interesting in the apache logs:

ap_getword_white failed, referer: http://intranet.olcc.com/zope/Plone/Members/pvecchio/index_html/document_view

missing/corrupt NTLM header, referer: http://intranet.olcc.com/zope/Plone/Members/pvecchio/index_html/document_view

And the Plone's error log shows this:

User Name (User Id) Anonymous User (None)

Still wonder if ntlm isn't failing somewhere. But why only on certain buttons (requests)? That circle of thinking keeps making me believe you are right about the roles/permissions thing. If so, it'd be easy to run an external script that would change the member roles, and iterate through all the members.

Let me know if the above error makes any sense to you.

Nice work. I hope you'd consider turning your work over to the GRUF team so that it could be incorporated into all future versions. But it's your work and your call.

It seems that whenever a user doesn't have the correct permissions, authentication falls back to basic. That's why you keep getting this authentication dialog. Yesterday, I found out that RemoteUserFolder doesn't correctly assign roles to automatically created users (the domain roles and '*' user thingy). This means that these users will only be assigned the implicit "Authenticated" role which might not have the permissions to edit their preferences. Can you verify that?

Also, when I click cancel in that basic auth dialog, I stay authorized as the domain user and I get a verbose Plone error message that lists which roles I'm assigned to and which permission were needed to perform the requested operation (for you it's probably the 'edit prefs' and for me it's the 'list folder contents' permission). I think I get the message because I have the 'Verbose Security' product installed.

When I (as the Zope manager) manually assign the necessary roles to the domain user (I can do it in either GRUF's Users tab or in RUF) and then repeat the operation as the domain user, it is performed and no basic auth dialog appears.

I will update my patch with to the modifications you suggested and see if it works with GRUF 3.3. We invested so much time in this, it would be a shame if we couldn't get it to work, don't you think? ;-)

** UPDATED 08-02-2005 **

I applied the patch code to GRUF 3.3 - here's the only thing you have to do to make it work:

return GRUFUser.GRUFUser(u, self, isGroup = 0, source_id = src.getUserSourceId()).__of__(self)

Just had to add the other attributes as it is written in 3.3. Works for me like it does in 2.0.1.

I'm still having those small issues asking me to authenticate again in 'my preferences' and such. Here's the pieces of the error msg I've been able to locate:

Module ZPublisher.Publish, line 92, in publish
Module ZPublisher.BaseRequest, line 449, in traverse
Module ZPublisher.HTTPResponse, line 671, in unauthorized

Time 2005/08/02 15:06:19.633 GMT-4
User Name (User Id) Anonymous User (None)
Request URL http://intranet.olcc.com/zope/Plone/Members/pvecchio/index_html/metadata_edit_form
Exception Type Unauthorized
Exception Value You are not authorized to access this resource.

Username and password are not correct.

TraversalRequestNameStack []
URL 'http://intranet.olcc.com/zope/Plone/Members/pvecchio/index_html/metadata_edit_form'

AUTH_TYPE 'NTLM'

** Here's the patch error: **

[root@xxxx GroupUserFolder]# patch -b < GroupUserFolder.py.patch
patching file GroupUserFolder.py
Hunk #1 FAILED at 41.
Hunk #2 FAILED at 448.
2 out of 2 hunks FAILED -- saving rejects to file GroupUserFolder.py.rej

** and the reject file itself **

***************
*** 41,46 ****

from interfaces.IUserFolder import IUserFolder

## Developers notes
##
## The REQUEST.GRUF_PROBLEM variable is defined whenever GRUF encounters
--- 41,48 ----

from interfaces.IUserFolder import IUserFolder

+ from AccessControl.User import _noroles;
+
## Developers notes
##
## The REQUEST.GRUF_PROBLEM variable is defined whenever GRUF encounters
***************
*** 446,455 ****
# => we refuse authentication
return None

- ## ## I DON'T KNOW IF WE HAVE TO PASS VALIDATE
- ## def validate(self, request, auth='', roles=_noroles):
- ## return self.Users.validate(request, auth, roles)
-

# -----------------------------
# Private User Folder interface
--- 448,468 ----
# => we refuse authentication
return None

+ ## Addition by Hannes, www.hannesschmidt.de
+ ##
+ ## Override validate() as well. This fixes the problem that RemoteUserFolder can't be used as a
+ ## source to GRUF. This method was implemented in analogy to authenticate(), i.e. iterate users
+ ## first user that validates is returned, if no user validates, base class' validate() is
+ ## invoked.
+ ##
+ def validate( self, request, auth='', roles=_noroles ):
+ for src in self.listUserSources():
+ u = src.validate( request, auth, roles );
+ if u is not None:
+ return GRUFUser.GRUFUser(u, self,).__of__(self)
+ return AccessControl.User.BasicUserFolder.validate( self, request, auth, roles )
+ ##
+ ## End of addition by Hannes

# -----------------------------
# Private User Folder interface

***************************************

I've found that the problem I'm getting with 'my preferences' and 'edit' are that it asks me for authentication. The auth box asks about olcc.intranet.com where the site is olcc.intranet.com/zope/Plone. Could that be part of it?

Although I have a similar setup to you, everything seems to work ok for me, including preferences and creating/editing items. Excuse me if I'm asking the obvious: Did you restart Zope after modifying the sources? Can you send me patch's output so I can see why it failed? Did you set roles and permissions (security tab) correctly? Try accessing the plone site through Zope's HTTP port as the Zope manager and bypassing NTLM auth. Unless you disabled Zope's HTTP port as described in Appendix 3, the plone site can be reached with basic auth at http://<your server>:8080/zope/<your plone site id>.

FYI, this is my setup:

Archetypes (Installed product Archetypes (1.2.5-rc5))
BTreeFolder2 (Installed product BTreeFolder2 (BTreeFolder2-1.0.1))
CMFActionIcons (Installed product CMFActionIcons (0.9))
CMFCalendar (Installed product CMFCalendar (CMF-1.4.7))
CMFCore (Installed product CMFCore (CMF-1.4.7))
CMFDefault (Installed product CMFDefault (CMF-1.4.7))
CMFFormController (Installed product CMFFormController (1.0.4))
CMFPlone (Installed product CMFPlone (2.0.5))
CMFQuickInstallerTool (Installed product CMFQuickInstallerTool (1.5.1))
CMFTopic (Installed product CMFTopic (CMF-1.4.7))
DCWorkflow (Installed product DCWorkflow (CMF-1.4.7))
Epoz (Installed product Epoz (0.8.5))
ExternalEditor (Installed product ExternalEditor (0.8))
ExternalMethod (Installed product ExternalMethod (External Method-1-0-0))
Formulator (Installed product Formulator (Formulator 1.6.2))
GroupUserFolder (Installed product GroupUserFolder (2.0.1))
MIMETools (Installed product MIMETools)
MailHost (Installed product MailHost (MailHost-1-3-0))
OFSP (Installed product OFSP (OFSP-1-0-0))
PageTemplates (Installed product PageTemplates (PageTemplates-1-4-0))
PlacelessTranslationService (Installed product PlacelessTranslationService (1.0))
PloneErrorReporting (Installed product PloneErrorReporting (0.11))
PluginIndexes (Installed product PluginIndexes)
PortalTransforms (Installed product PortalTransforms (1.0.4))
PythonScripts (Installed product PythonScripts (PythonScripts-2-0-0))
RemoteUserFolder (Installed product RemoteUserFolder)
Sessions (Installed product Sessions)
SiteAccess (Installed product SiteAccess (SiteAccess-2-0-0))
SiteErrorLog (Installed product SiteErrorLog)
StandardCacheManagers (Installed product StandardCacheManagers (StandardCacheManagers-1-1-0))
TemporaryFolder (Installed product TemporaryFolder)
Transience (Installed product Transience)
VerboseSecurity (Installed product VerboseSecurity (VerboseSecurity-0.6))
ZCTextIndex (Installed product ZCTextIndex)
ZCatalog (Installed product ZCatalog (ZCatalog-2-2-0))
ZGadflyDA (Installed product ZGadflyDA)
ZODBMountPoint (Installed product ZODBMountPoint)
ZReST (Installed product ZReST (1.1))
ZSQLMethods (Installed product ZSQLMethods)
ZopeTutorial (Installed product ZopeTutorial (Zope Tutorial 1.1))
generator (Installed product generator (1.0.1))
validation (Installed product validation (1.0.1)) 

Hannes. Great work. Yes the patch works for me. I had to cut/paste the changes though. Cmd line patch apply failed. The login works, as does the sharing tab. But it appears to break other things. My Preferences wouldn't open. Add New Item adds an item, but you can't edit it.

I pasted the code in the new GRUF 3.3 (used 2.0.1 for the initial trial above). Had to paste into line 45 and line 1045 to be consistent with your placement. 3.3 came up with a site error right away (although it did authenticate me). Error - _init_() takes exactly 5 arguments (3 given). Thought you'd like to know that.

FYI - I'm using Plone 2.0.5.

peppiv

The links were very insightful. Take a look at the update to Appendix 5. I'd be interested to know if the patch works for you. Thank you very much for your support!

-- Hannes

I was very suspect when the GRUF folks said they had it working. I've never read of anyone getting it this far. Even at the Plone Conference in New Orleans last week.
Please post anything (even if it's not 100%). I've been dying over this for the last few weeks. You can always email me the GRUF source hacks. I'd love to see them and see if I can help.
I had always thought the fault was with RUF and its unpersistant use of the UserFolder. Seems like you've found something.
Congratulations!

Also, I found this link: http://sourceforge.net/mailarchive/forum.php?thread_id=5428110&forum_id=8090

and this link: http://sourceforge.net/mailarchive/message.php?msg_id=9132433 This comes from Dylan Jay - the author of RemoteUserFolder.

Might be helpful.

It appears that GRUF doesn't delegate validate() to any of its sources. The method validate(), OTOH, is the work horse of RemoteUserFolder. Given that GRUF doesn't call validate() at all it is obvious that RUF can't be used as a source for GRUF. I find it a bit strange that the GRUF folks told you that they got RUF working with GRUF.

Anyway, I hacked the GRUF sources a bit and I can now authenticate with RUF as a source of GRUF. There are still problems. I do not completely understand what validate() is for and when it is called. I just wanted to let you know that I'm working on it ...

I followed the how-to and it worked easily. The only problem I have is with GRUF. Without GRUF working, the sharing tab and obviously, groups does not work. This is a deal killer for me. I've been emailing the team from GRUF to see what they think, and they've responded that they've had cases where GRUF and RUF work together. Not so for me. Does anyone have any ideas how to fix this?
If I use RUF, it works except for GRUF. If I use GRUF and use RUF as a user source, it does not authenticate. I cannot track down the problem. There are no errors I can find.
peppiv