20 April 2010 16:02 -
Today I made a small and simple script to make my crontabs more readable. The
usage is quite simple. Run te script and redirect the output to a file.
Review this file. Then when the file is ok,
load it with crontab <file>
The script:
#! /usr/bin/env bash
crontab -l | while read min hour day month wday regel
do
if [ x${min:0:1} = "x#" ]
then
echo "${min}" "${hour}" "${day}" "${month}" "${wday}" "${regel}"
else
if [ x${min:0:1} = "x" ]
then
echo
else
printf "%-7s %-7s %-7s %-7s %-7s %s" "${min}" \
"${hour}" "${day}" "${month}" "${wday}" "${regel}"
echo
fi
fi
done | sed 's/[ ]*$//'
13 April 2010 20:16 -
Finally I have some time again to write a blogpost. February and march were busy months in which
I moved from Enschede to Delft. Main advantage of this is of course that I live closer to my
work, which on average safes me around 12 hours a week of travel time. Another advantage
is of course that the house is larger (around 30 m^2) and do not have to park my car 10 minutes
away.
But that was not all that happened in the last 2 months. I also passed the PRINCE 2 Foundation
exam on the 19th of february.
Beside the PRINCE 2 Foundation exam I also received some Novell certification.
Due to my LPIC certifications I became a Novell Certified Linux Administrator and
a Novell Data Center Technical Specialist.
16 April 2009 18:54 -
Most people are familiar with using some kind of naive bayesian filtering to
classify mail as spam or non-spam. Or at least they use a spam filter that uses
bayesian filtering without knowing that it uses bayesian filtering. Nothing new
so far. But beside spam I had another criteria for classifying email: How
important is it to me? In my opinion there are two kinds of mail besides of
course spam email.
The first being mail that needs no immediate action. In example linkedin invites, the
mail with the link to that funny youtube movie, mailstatistics from servers,
updates of who started following you on twitter etcetera.
The second is of course email that needs immediate action. In example the mail
from nagios that your raid 1 mirror is degraded or the invitation to that cool
party. For me this is email I want to see when I am travelling and can not wait
till I am at my desk again.
Therefor I wanted to classify every incoming none spam mail into the important
or unimportant category. Hence I decided to try classifying my email this way
with bayesian filtering. The idea was that bayesian filtering should save me
the overhead of setting up complex rules.
Setting this up was straight forward. First I divided all regular email of the
last 2 months in 2 categories: important and unimportant. This gave me the
training sets to start training my filter. After this I trained the filter and
then configured the filter to place all unimportant email in another 'later'
mailbox folder, instead of putting it in the default Inbox.
Now the only challenge was making sure classification mistakes gets corrected. This is done by
archiving every mail in one of two archive mailboxes. In my case '2009l' or '2009n'. Every night
I run a scheduled job to retrain the filter on these archives.
An interesting question after all this is: Does it work? In short yes, it does work. Around 90%
of my incoming email is classified correctly and I did not miss any cool party as far as I know.
03 March 2009 18:11 -
During the last past months I have migrated from a FreeBSD 5.2 server to a FreeBSD 7.1 server.
This also meant I had to migrate my jails to the new FreeBSD 7.1 machine. To save time compiling
the same ports over and over again I created a package building jail, which contains
all the generic packages for my jails. In this post a quick overview of the settings I made
to get this working.
In the build jail I first installed portupgrade from the ports to make it easier to upgrade ports.
After this I made the following setting in /etc/make.conf.
PACKAGES=/packages
This indicates that all packages should be created in the /packages directory. That being the
location where I keep my packages on this host. Besides setting this in /etc/make.conf I also
have to set this in /usr/local/etc/pkgtools.conf. This can be done by making sure
the PACKAGES environment variable is set in this file. I.e.
ENV['PACKAGES'] ||= '/packages'
After this I just had to install all the base packages with make package-recursive.
Upgrading of the packages can now be done with portupgrade -vrRap to make sure
packages are being build during the upgrade process.
To allow the other jails and other machine I have exported the /packages directory on
the build host as http://buildhost/packages
Configuring the clients is now straight forward in fact. In the
/usr/local/etc/pkgtools.conf on the other jails I just had to set
the PKG_SITES variable. I.e.
PKG_SITES = [ 'http://buildhost/packages/', ]
After this upgrading of the packages on the client jails can easily be done
with portupgrade -avRrP. Or you can set the USE_PKGS_ONLY
option in the pkgtools.conf file to specify the packages that only
should be installed/upgraded from a package.
25 February 2009 08:14 -
The sequel of my setup of openldap on FreeBSD 7.1. This time the configuration of the clients.
If you want to read the short introduction to setting up a server, then you can find
it
here
The first thing we need to do is install the necessary packages. For the configuration
described here we need at least the following (or newer versions of course):
nss_ldap-1.264_1
openldap-client-2.4.13
pam_ldap-1.8.4_1
pam_mkhomedir-0.1
sudo-1.6.9.20
After installation of these packages we can start to configure everything. First we have to make a
/usr/local/etc/ldap.conf. Mine contains the following lines:
host hildr.jeroen.se
base dc=jeroen,dc=se
uri ldaps://hildr.jeroen.se/
binddn uid=nss,dc=jeroen,dc=se
bindpw xxxxxxxxxx
rootbinddn cn=Manager,dc=jeroen,dc=se
port 389
pam_check_host_attr yes
nss_base_passwd ou=People,dc=jeroen,dc=se
nss_base_shadow ou=People,dc=jeroen,dc=se
nss_base_group ou=Group,dc=jeroen,dc=se
ssl on
tls_checkpeer no
tls_reqcert allow
tls_ciphers HIGH:MEDIUM:-SSLv2
sudoers_base ou=SUDOers,dc=jeroen,dc=se
In the configuration above nss is the user that will be used for connecting to the ldap server. This
user should exist in LDAP of course. We also have to set the appropriate password by the bindpw option.
And do not forget to change the other options to fit your situation as well.
Now we have created the ldap.conf file, we link
/usr/local/etc/nss_ldap.conf to /usr/local/etc/ldap.conf.
rm /usr/local/etc/nss_ldap.conf
ln -s /usr/local/etc/ldap.conf /usr/local/etc/nss_ldap.conf
We also have to
create a /usr/local/etc/ldap.secret file which normally contains the password of the
root user for connecting to the ldap. Because we will not be connecting as root user to
our ldap server from the clients it is enough to run the following commands:
touch /usr/local/etc/ldap.secret
chmod 600 /usr/local/etc/ldap.secret
Believe it or not, but we are almost done configuring already. We just have to make sure
that the directory where the user home directories are being placed exists. After that we have
to edit 3 files. First /etc/nsswitch.conf, which should look like this:
group: files ldap
group_compat: nis
hosts: files dns
networks: files
passwd: files ldap
passwd_compat: nis
shells: files
services: compat
services_compat: nis
protocols: files
rpc: files
Then /etc/pam.d/sshd, which should look like this:
# auth
auth sufficient pam_opie.so no_warn no_fake_prompts
auth requisite pam_opieaccess.so no_warn allow_local
#auth sufficient pam_krb5.so no_warn try_first_pass
#auth sufficient pam_ssh.so no_warn try_first_pass
auth sufficient /usr/local/lib/pam_ldap.so no_warn try_first_pass ignore_unknown_user
auth required pam_unix.so no_warn try_first_pass
# account
account required pam_nologin.so
#account required pam_krb5.so
account required pam_login_access.so
account required /usr/local/lib/pam_ldap.so ignore_unknown_user
account required pam_unix.so
# session
#session optional pam_ssh.so
session required /usr/local/lib/pam_mkhomedir.so debug umask=0077 skel=/usr/local/share/skel
session required pam_permit.so
# password
#password sufficient pam_krb5.so no_warn try_first_pass
password required pam_unix.so no_warn try_first_pass
And finally /etc/pam.d/system
# auth
auth sufficient pam_opie.so no_warn no_fake_prompts
auth requisite pam_opieaccess.so no_warn allow_local
#auth sufficient pam_krb5.so no_warn try_first_pass
#auth sufficient pam_ssh.so no_warn try_first_pass
auth sufficient /usr/local/lib/pam_ldap.so no_warn try_first_pass
auth required pam_unix.so no_warn try_first_pass nullok
# account
#account required pam_krb5.so
account required pam_login_access.so
account required /usr/local/lib/pam_ldap.so ignore_unknown_user
account required pam_unix.so
# session
#session optional pam_ssh.so
session required pam_lastlog.so no_fail
session required /usr/local/lib/pam_mkhomedir.so
# password
#password sufficient pam_krb5.so no_warn try_first_pass
password required pam_unix.so no_warn try_first_pass
After all this we should be able to login with the ldap user accounts on the client. Assuming of
course the user is correctly configured in the OpenLDAP server.
17 February 2009 08:37 -
Today a small blog post to explain how I have setup my openldap server at home for authentication.
It is not an in detail explanation, but should give a good start if you can do some
thinking yourself.
For this setup I had the following requirements:
- The data between the ldap server and the client should be encrypted.
- Access should be restricted to given hosts. That is a user which has acces on machine A
may not have acces on machine B by default.
- Sudo rights should also be managed by openldap.
First I had to install the openldap-server port with:
cd /usr/ports/net/openldap23-server
make install clean
And of course I had to enable it in /etc/rc.conf by adding the lines:
slapd_enable="YES"
slapd_flags='-h "ldapi://%2fvar%2frun%2fopenldap%2fldapi/ ldaps://192.168.0.80/ ldap://127.0.0.1/"'
slapd_sockets="/var/run/openldap/ldapi"
Then I had to edit my /usr/local/etc/openldap/slapd.conf configuration file:
include /usr/local/etc/openldap/schema/core.schema
include /usr/local/etc/openldap/schema/cosine.schema
include /usr/local/etc/openldap/schema/inetorgperson.schema
include /usr/local/etc/openldap/schema/ldapns.schema
include /usr/local/etc/openldap/schema/nis.schema
include /usr/local/etc/openldap/schema/misc.schema
include /usr/local/etc/openldap/schema/sudo.schema
The above code includes all the necessary ldap schemes. It should be noted that the sudo.schema
should be taken from the installation of the sudo port and copied to the openldap/schema directory.
The ldapns.schema is needed for limiting of users to hosts. This schema can be found
here
The next settings we need to set in the slapd.conf are the access rights. I use 2 user to
do more than authentication against the ldap server: Manager and nss. Which gives me the
following access settings:
access to attrs=userPassword
by dn="cn=Manager,dc=jeroen,dc=se" write
by anonymous auth
by * none
access to *
by dn="cn=Manager,dc=jeroen,dc=se" write
by dn="uid=nss,dc=jeroen,dc=se" read
by anonymous auth
And the last settings I had to make in the slapd.conf file where the
initial password for the root user (Manager):
rootdn "cn=Manager,dc=jeroen,dc=se"
rootpw {SSHA}9cyD+AfBLx0jvSSL7iJ7tMMB4hl2mN0P
And the settings for SSL, which assumes your keys are in /usr/local/etc/openldap/ssl/server.pem:
#TLSCipherSuite HIGH:MEDIUM:-SSLv2
TLSCACertificateFile /usr/local/etc/openldap/ssl/server.pem
TLSCertificateFile /usr/local/etc/openldap/ssl/server.pem
TLSCertificateKeyFile /usr/local/etc/openldap/ssl/server.pem
TLSVerifyClient never
The rest of the settings in slapd.conf were left to their default value. So I then started slapd with
/usr/local/etc/rc.d/slapd start
That is basically all you have to do to get slapd running. I used the following ldif file
to get the intial data into the ldap server:
dn: dc=jeroen,dc=se
objectclass: dcObject
objectclass: organization
o: jeroen dot se
dc: jeroen
dn: cn=Manager,dc=jeroen,dc=se
objectclass: organizationalRole
cn: Manager
After this I used phpldapadmin to fill my ldap server with more data. I will
post the configuration I use on the clients in the coming days.
11 February 2009 18:46 -
If you are like me administrating lots of unix servers where you have to do
different tasks at different servers simultaneously then you can easily lose track of which
screen window is doing what on your laptop. My solution for this in the zsh
shell is using the following in my .zshrc:
preexec()
{
if echo ${TERMCAP} | grep screen > /dev/null
then
screen -X title ${1}
fi
}
precmd()
{
if echo ${TERMCAP} | grep screen > /dev/null
then
screen -X title $(echo $PWD | sed "s#$HOME#~#")
fi
}
This sets the title of a window where a command is running to the name of the
command being run. When no command is running it sets it to the current
working directory. The advantage is that I no longer have to set the titles manually.
The disadvantage is that the titles can get very long. Therefor I have in my .screenrc
the following line:
bind l windowlist
Which allows me to quickly get the window list with ^Al.
06 February 2009 08:11 -
This week I learned a nice trick with brace expansion when using the bash or
zsh shell from a colleague from me. I already used brace expansion in simple
for loops like:
for i in {1..5}
do
echo $i
done
But as he demonstrated, you can also use it in constructs like:
cp file{,.20090206}
Which expands to
cp file file.20090206
04 February 2009 08:09 -
Last week I thought a little about naming volumegroups on SANs, logical volumes,
hosts and (micro)SD cards. They all have one thing in common, if you have a lot
of them you hardly can avoid numbering them. The only problem with this is of course
small typos that can lead to disasters. Fortunately there are of course solutions for this.
After some thinking I thought about using checksums in the numbering, which makes
it harder to create a disaster with just one typo. The best scheme I could find
to work for this was using the ISBN numbering scheme.
So i.e. I now number my (micro)SD cards as follows:
- jni-60001
- jni-10002
- ...
- jni-12761
- jni-72762
- jni-22763
- etc.
As you probably can see the first number is the checksum digit. A number is
valid if and only if the checksum digit + 2 times the second number + 3 times
the third number + 4 times the fourth number ... modulo 11 = 0. Note that the checksum digit
can also be X to indicate a value of 10.
If you look at the jni-60001 number. The checksum is 6. And the 1 is multiplied by 5. Summarized
this makes 11, which modulo 11 is of course 0. So this is a correct number. But what if we make
a typo i.e.:
- jni-60010, this becomes (6+4) modulo 11 = 10, which is not valid.
- jni-60002, this becomes (6+10) modulo 11 = 5 , which is not valid.
- jni-60019, this becomes (6+4+45) modulo 11 = 0 , which is valid. But then you already have
made 2 typos.
Of course I created a script for this that does all the hard work. It can be found
here.
19 January 2009 08:52 -
The first public beta of my jniatd script is available. jniatd is a perl based
proof of concept for a class based at daemon. Which means that a job will run
only when the specified time has passed and the class it belongs to evaluates
to true.
11 August 2008 12:59 -
While migrating my account to another machine which uses openldap authentication I ran
into the problem that cron were not working on the new machine. A quick look in the
logs revealed the following error:
Aug 11 08:52:00 shell /usr/sbin/cron[82693]: (jnieuwen) ORPHAN (no passwd entry)
Some googling did not give any usefull results, but was quite clear
to me that it had to do with a openldap or pam configuration issue. Taking a quick look
at my /etc/pam.d/cron I noticed that no openldap config was in that file. Hence
I added
account sufficient /usr/local/lib/pam_ldap.so
And after a restart of the cron daemon everything was working again.
22 January 2008 18:27 -
Sometimes you find yourself doing a lot of things at a shell prompt which you have to
do on another host later on. So what you want is a script. Yes really. All according
to the motto: Think once,
type once, execute often.
So time to export your command history to a file as a starting point for a
script. The easiest way I found for this is: fc -n -l -100.
Where 100 is the number of lines you need from your history.
19 December 2007 16:57 -
I passed the Red Hat Certified Engineer exam last Friday.
19 November 2007 18:10 -
Last week I ran in to the problem that my svntag.sh script should run
on the first of every month. Unfortunately my svntag.sh script misses all
kind of PATH info so it will probably not run as a cronjob without
modifications.
Being to lazy to check and fix the possible problems while running from
cron`,' I came up with:
for i in {0..24}
do
echo svntag.sh | at -t $(date -j -v+${i}m 200712010407 +%Y%m%d%H%M)
done
Which works when using the zsh shell on FreeBSD.
07 November 2007 13:37 -
Last week I upgraded to Mac OS X Leopard from Tiger. Having a backup of my important
files I simply choose the upgrade option. And 1.5 hours later or so I had a
working Leopard install.
My main problem with Leopard is that X11 has changed dramatically. So much that I
have real problems using wmii-3 for xterms.
The improvements in Terminal.app saved Leopard from a rollback to Tiger because of X11.
Terminall.app has become usable now. Especially after I issued the
defaults write com.apple.Terminal FocusFollowsMouse -string YES
command.
I also quickly decided to put the dock back to something reasonable with the
defaults write com.apple.dock no-glass -boolean YES
killall Dock
commands.
12 October 2007 17:03 -
Yesterday the harddisk of astrild.jeroen.se broke down. Unfortunately this also means that
my scanning of bluetooth devices in and around my house has come to a halt.
18 September 2007 17:52 -
Last weekend was the
EuroBSDCon 2007 in Copenhagen.
Tuesday
Traveling by car from The Netherlands to Copenhagen.
Wednesday
I visited the PF and Virtualized networkstack tutorials on wednesday.
Especially the Virtualized networkstack tutorial was interesting.
Thursday
On thursday I had no real plans so I explored Copenhagen by bike. I should note that Copenhagen
is a really nice city which is worth paying a visit.
Friday
Visited the opening session. After that I went to the talk of Sam Leffler with
Robert Watson as the virtual Sam. This because Sam was not at the conference
due to a flu. After that I visited the talk of Ryan Bickhart about SCTP. This
learned me what SCTP is and what it can do. Interesting talk in which I really
learned something new. Then it was lunch time. Which was followed by a talk of
Robert Watson about the FreeBSD security features. Good talk. Simon Nielsen
talked about the FreeBSD security officer function. His talk was followed by
Claudio Jeker which showed how OpenBSD can be used as a routing platform.
Steven Murdoch closed the day by introducing the fingerprinting of hosts using
clockskew. A very interesting story.
Saturday
Saturday I started the day with breakfast and when I arrived at the conference
location I was just in time to listen to a BSD succes story told by Yvan
Vanhullebus. Then it was time for the keynote by John Hartmann about pipe
implementation on mainframes. I took a very long break after that went then to
the humerous talk of Pawel Dawidek about ZFS. If you have not taken a look at
ZFS already you should really do so. The last talk of the conference I went to
was about network protocol testing given by George Neville-Neil. The conference
ended with a 3rd party software deathmatch which was won by debian by bribing the
judges with a cake.
Sunday
Traveling back to The Netherlands by car. Took me 3 hours longer than expected
due to the heavy traffic in Germany.
10 August 2007 21:07 -
Last week I updated the patch for the
openospfd-devel port problem with point to point links
to work with FreeBSD current. The old patch did not work on current because GCC 4 is used on current.
The new patch can be found on the PR page linked above.
11 June 2007 10:05 -
After passing the Sun Solaris CX-310-202 exam on the 4th of June`,' I received
today the offical letter that states that I have passed all the requirements for the
Sun Solaris 10 System Administration Certification.
07 June 2007 07:51 -
zsh has another nice feature I use to display the latest hilights on irc.
The
zsh option I use for this is the
periodic function and the
PERIOD
environment variable. First off all I have set
PERIOD to 60 in my
~/.zshrc
config file. This ensures that the periodic function is executed before
a new prompt if at least 60 seconds have passed.
The periodic function is defined as:
function periodic() {
diff $HOME/.periodic/last_hilight $HOME/.periodic/last_hilight.old | grep "^<" |
cut -f2- -d: | sed -e "s/:/ /" -e "s/^/IRC: /" && cp $HOME/.periodic/last_hilight $HOM
E/.periodic/last_hilight.old
}
Which means that it compares the $HOME/.periodic/last_hilight file with the
$HOME/.periodic/last_hiligt.old file and displays the difference.
You might wonder how the last hilight from irc now comes in the
$HOME/.periodic/last_hilight file. This is accomplished by an irssi plugin I
extended. More on this plugin later because I still have to document this extension.
05 June 2007 07:29 -
Sunday I wrote about configuring your prompt in
zsh and I promised to write
about configuring the more advanced commandline completions in
zsh. This is
in fact quite simple. Adding
autoload -U compinit
compinit to
~/.zshrc
takes care of most of it. However I also wanted command line completion for
hosts not in my
/etc/hosts file and for usernames I have on other
hosts/networks.
To extend the username completion you have to add the usernames to your
~/.zshrc with the following construct:
zstyle ":completion:*" users jnieuwen root
where jnieuwen and root are of course the usernames you want autocompletion on.
The construct for the hostname completion is almost the same:
zstyle ":completion:*" hosts alberich.jeroen.se hermod.jeroen.se
03 June 2007 13:49 -
Last friday I read something about the better completion features of
zsh
compared to bash. So I gave
zsh a try. In
zsh I could import and use
my
bash aliases so the only two things left to do was configure a usefull
prompt and getting the command line completion to work.
In
zsh you have 2 main prompts: PS1 and RPS1. PS1 has the same function as
in
bash. RPS1 is an extra prompt which will be displayed on the right-hand side
of the screen. I decided to set RPS1 to the current time. And after some thinking I
had my prompts ready which look like:
(~blog) (jobs: 0)
[1004][jnieuwen@idunn]% 13:57:33
This prompt is generated with the following code:
export RPS1="%D{%H:%M:%S}"
export PS1="(%4~) (jobs: %j)
[%!][%{${fg[blue]}%}%B%n%{${fg[yellow]}%}@%{${fg[green]}%}%m%b%{${fg[default]}%}]%# "
Later this week more about configuring the commandline completion.
29 May 2007 17:24 -
This weekend I had the problem that I had to check several hosts on a network
behind NAT with nagios. One solution would be to add portforwardings on the NAT
firewall for each host and use check_nrpe with different port settings from the
nagios machine. This is of course a maintenance nightmare.
The solution that I choose was therefor a little different. I installed nrpe
server and the nrpe plugins on all hosts. Then a portforwarding to port 5666 of
one host (irobot) was added in the firewall. In the nrpe config of irobot I
added the checks for irobot and checks for the other hosts by adding lines like
these to the nrpe.cfg of irobot:
cammand[vacdepot_fs_root]=/usr/lib/nagios/plugins/check_nrpe -H vacdepot -c check_fs_root
In the nrpe.cfg of vacdepot check_fs_root is defined of course.
So I am using nested nrpe calls to get the data I want. The advantage being
that I do not have to have a portforwaring for every host I want to monitor
behind the NAT firewall.
28 May 2007 12:34 -
Sometimes you will find that in bash you have to do a substitution in the value
of a variable. i.e. in my blogging scripts I have the filename in the variable
POSTFILE and for this posting POSTFILE has the value of 2007-05-28-12-34_string_replacements_in_bash.m4.
But when I have to generate a link to the post it should end in
.php
instead of
.m4. The naive approach would be to do something like:
echo $POSTFILE | sed "s/\.m4$/\.php/"
Bash however has a more elegant solution for this:
echo ${POSTFILE/%\.m4/.php}
Where % means to match at the end of the
POSTFILE variable. More
information about the syntax can be found in the bash man page in the Parameter
Expansion section.
