source: trunk/third/moira/afssync/INSTRUCTIONS @ 23095

Revision 23095, 7.2 KB checked in by ghudson, 16 years ago (diff)
Import the moira package from SIPB Debathena.
1The executables are in /moira/bin/ on the moira server, with sources
2in /mit/moiradev/src/afssync/.  Most of the commands are run on the
3Moira server.
5####    Set up a workspace                                              ####
7mkdir -p /moira/sync
8cd /moira/sync
10####    This is preparation for the resync, to save non-Moira users.    ####
11First, get a recent copy of the prdb, and extract non-Moira entries:
13        /moira/bin/udebug prill -port 7002
14        rcp -px root@prill:/usr/afs/db/prdb.DB0 prdb.old
15        /moira/bin/udebug prill -port 7002
16If the two udebugs show that the version changed, lather-rinse-repeat.
17(udebug can be found in /usr/athena/bin; "prill" here and below is some
18DB server)
19(Also check for "0 of them for write" at the end.  It might matter.)
21        /moira/bin/pt_util -x -m -u -g -d prdb.extra -p prdb.old
22        perl /moira/bin/ < prdb.extra > prdb.extra.sort
23to extract and prepare the personal groups and special user entries in
24the old prdb for being reincorporated into the new prdb.
26        awk -F\| '$9 == 3 {print $1}' /backup/backup_1/users > /tmp/deactivated
28and the following perl script:
30#!/usr/athena/bin/perl -w
32open(OUT, ">prdb.extra.trimmed");
34for ( `cat /tmp/deactivated` ) {
35    chop;
36    $ex{$_} = 1;
39$punt = 0;
41foreach $L ( `cat prdb.extra.sort` ) {
42    @w = split(/ /,$L);
43    $_ = $w[0];
44    if ( /:/ ) {
45        @x = split(/:/,$w[0]);
46        if ($ex{$x[0]}) {
47            $punt=1;
48        } else {
49            $punt=0;
50        }
51    } else {
52        # If we got here, we're either a user, a prefixless
53        # group, or a group member.
54        $punt = 0 if $w[0];
55    }
56    print OUT $L unless $punt == 1;
60exit 0;
62to remove the personal groups for users who are deactivated
64        awk '/^[^ ][^:]*@/ {printf "KERBEROS:%s\n",$1}' prdb.extra.trimmed \
65                > foreign
66        blanche afs-foreign-users -f foreign
67Get a list of all the type (non- cell)
68users, and sync the Moira list afs-foreign-users to this list.
69Moira then adds those entries to the group system:afs-foreign-users,
70thus keeping them from being lost in the prdb resync.
71Sanity checking the diffs before running the blanche command is recommended.
73        awk '/^[^ 0-9][^:@]*$/ {printf "KERBEROS:%s@ATHENA.MIT.EDU\n",$1}' \
74                prdb.extra.trimmed > oddities
75        awk '/^[^ ][0-9.]* .*$/ {printf "KERBEROS:%s\n",$1}' prdb.extra.trimmed\
76                 >> oddities
77        echo "LIST:afs-foreign-users" >> oddities
78        blanche afs-odd-entities -f oddities
79Do the equivalent of afs-foreign-users for domestic users.  We make
80the afs-foreign-users list a member of the more general afs-odd-entities.
81Sanity checking the diffs before running the blanche command is recommended.
83WAIT for the incremental updates from the `blanche` changes to complete.
85####   Now the actual resync begins.  Incremental updates must stop.   ####
87        touch /moira/afs/noafs
88to disable AFS incremental updates during the synchronization.  The
89afs.incr (?) will wait 30 minutes on an incremental update before
90timing out, so the resync should complete in that time, or list
91changes in Moira might need to be propagated by hand.
93        /moira/bin/afssync prdb.moira
94to dump the prdb data that is in Moira (users, groups, and group
95memberships).  This step takes about ten minutes, but can be done
96concurrently with the next few steps.
98REPEAT the above commands, thus regenerating prdb.trimmed from a now
99completely-up-to-date prdb.
101*** Make sure the "afssync" command has completed ***
103        cp prdb.moira
104        /moira/bin/pt_util -w -d prdb.extra.trimmed -p \
105                >& prdb.extra.err
106This use of pt_util will presumably log errors about failed user
107creations and list additions.  (To start over, do both the `cp` and
108`pt_util` again.)  You can filter out the "User or group doesn't exist"
109type of lines that were caused by a user deactivation with something
111        awk -F\| '$9 == 3 {print $1}' /backup/backup_1/users > /tmp/deactivated
112        perl -e 'for(`cat /tmp/deactivated`){ chop; $ex{$_}=1;} \
113                foreach $L (`cat prdb.extra.err`){ $f=0; \
114                @w=split(/[ :]/,$L); for(@w){ $f=1 if $ex{$_}; } \
115                next if $f; print $L; }'
116Now, back to the resync.
118The only remaining errors should be errors creating system:foo groups,
119be cause they already exist.  These generally mean that that group has
120an odd user on it (root instance, IP acl, etc.) and can safely be
123Errors of the form:
124Error while creating dcctdw:foo: Badly formed name (group prefix doesn't match owner?)
125are probably an indication that a user with personal groups had a
126username change (in the past they have also meant that a user with
127personal groups was deactivated and the uid was re-used (this was
128becasue we didn't trim the prdb.extra.sort file in the past.))
129Assuming htese errors are due to a username change, the groups should
130be renamed, and you should regenerate prdb.extra.trimmed starting with
131a fresh prdb from prill.  (You may want to abort and
132rm /moira/afs/noafs and try again later.)
134        pts listmax > prdb.listmax
135        foreach i ( <db servers> )
136            rsh $i -l root -x /bin/athena/detach -a     # detach packs
137            rsh $i -l root -x rm -f /usr/afs/db/{,pre-resync-prdb}
138            rcp -px root@${i}:/usr/afs/db/
139        end                                             # staging
140        foreach i ( <db servers> )
141            bos shutdown $i ptserver -wait
142            bos exec $i "mv /usr/afs/db/prdb.DB0 /usr/afs/db/pre-resync-prdb; rm /usr/afs/db/prdb.DB*; mv /usr/afs/db/ /usr/afs/db/prdb.DB0"
143        end
144        foreach i ( <db servers> )
145            bos restart $i ptserver
146        end
148        /moira/bin/udebug prill -port 7002
149to watch the status of the servers to make sure things are going well,
150where "prill" is preferred db server (the sync site).
152Make sure the beacons are working, and that once quorum is established
153(~90 seconds) that the servers are resynchronizing their notions of
154the databases and that the "dbcurrent" and "up" fields all become set
155and the state goes to "1f".  Also, if "sdi" isn't running, watch out
156for large rx packet queues on port 7002 using rxdebug, as the
157fileservers may get excessively backlogged, and restart servers, if
158necessary, if the congestion remains excessive.
160        pts listmax
161        cat prdb.listmax
162and if the id maxima are lower than the saved ones, reset them
163appropriately to the saved ones using `pts setmax`.
165        pts ex system:administrators
166as a good spot check, especially since it has special people.
167(also spot check one of the personal groups and perhaps, something like
168the membership of rcmd.reynelda)
170        rm /moira/afs/noafs
171to remove the lock file and let Moira's afs incrementals continue.
173        The afssync program doesn't deal with null instance KERBEROS
174members of lists which are groups (example: if LIST zacheiss contains
175KERBEROS zacheiss@ATHENA.MIT.EDU).  To get around this, run:
179Which will create /var/tmp/sync.out, which contains the pts commands
180needed to add all the null instance KERBEROS members back to the pts
181groups they belong in.  If it looks sane, run:
183sh /var/tmp/sync.out
185Any failed additions are probably from lists that contain both USER
186username and KERBEROS username@ATHENA.MIT.EDU.
1901. Don't do this when you're tired...  There may be no cleanup procedure
191available, with certain mistakes.
1932. /moira/afs/noafs is only good for 30 minutes.  Keep track of the
194critical log, and you may have to do some operations by hand when the
195operation is complete.  Also, if requests depend on other requests, they
196may be processed out of order, and fail, and may need to be done by hand.
Note: See TracBrowser for help on using the repository browser.