Main Files
Contents |
Essential Zentyal files for troubleshooting and tweaking
Introduction
The objective of the Zentyal GUI is to be an easy, yet powerful interface to perform all our system administration. Even then, is not always possible to do everything for everyone. In this document we are going to study the main configuration files and zentyal CLI commands for those who want to go a little deeper under the hood. Our goal will be the troubleshooting, lower level configuration review to gain better understanding on how Zentyal works and minor tweaks that we will be able to perform editing those files.
Zentyal specific configuration files
Let's start with the files located under /etc/zentyal, listing this directory, we will be able to see a bunch of .conf files
core.conf firewall.conf network.conf ppa.gpg zentyal.conf dns.conf hooks/ openvpn.conf pre-save/ events.conf logs.conf post-save/ users.conf
And also some directories (hooks, post-save, pre-save) related with customs hooks, which will be covered in a different how-to.
If you manually change this files, they won't be overwritten by default since dpkg will use the force-confold option by default (although you can manually replace them with the package maintainer's version if you want.
the basic instructions for the formatting of this .conf are contained at the beginning of the file:
# Everything after a '#' character is ignored # # All whitespace is ignored # # Config keys are set this way: # # key = value # # They may contain comments at the end: # # key = value # this is ignored
Let's review some of these configurations
File: core.conf
redis_port = 6380 # Redis is an in-memory key-value data store for the zentyal configuration (a configuration database to say it roughly), by default the redis server only listens in the loopback interface, here you can change in which port. widget_ignore_updates = yes # If you don't want to see the possible software updates in the Dashboard, you can choose 'no'. desktop_services_enabled = yes desktop_services_port = 6895 # Auto-configuration of desktop clients for the services provided by Zentyal (mail, samba, Jabber, VoIP, ...), is implemented as a webservice which listens on all interfaces (0.0.0.0:6895). The associated service ''Desktop Services'' is allowed by default in the firewall section ''Internal networks to Zentyal''.
File: firewall.conf
iptables_log_limit = 50 # Sometimes the firewall generates a huge amount of information, you can limit the maximum number of packages logged per minute. iptables_log_burst = 10 # You may also want to avoid logging a lot of times the exact same event, this will limit the bursts of identical packets appearing in your logs. iptables_log_drops = yes # Whether or not create logs of the dropped packets. nat_enabled = yes # The default behavior is to NAT the internal networks when redirecting to external, but we may have a router in charge of this, or we may want a different behavior, like different external address assigned to different internal networks.
File: network.conf
ifaces_to_ignore = sit,tun,tap,lo,irda,ppp,virbr,vboxnet,vnet # Types of interfaces that won't be displayed to be configured explicitly by Zentyal on the ''network'' section.
File: users.conf
mk_home = yes # whether to create user homes or not dir_umask = 0077 # default mode for home directory (umask mode) enable_quota = yes # enable quota support slave_time = 5 # number of minutes between master pushes of LDAP changes to its slaves multiple_ous = yes # This option allows you to have more than one organizational unit in the same ldap. This may be used, for example, to have multiple Zarafa domains.
File: zentyal.conf
debug = yes # Whether to enable the DEBUG> logs in ''/var/log/zentyal/zentyal.log'', this file will be commented below in this document. It's a good idea to enable it if we want to perform basic Zentyal troubleshooting. dump_exceptions = yes # Show full stack trace in the Zentyal interface when you hit a bug, also very interesting for diagnosis.
Zentyal output and debugging files
As we did in the first section, let's have a look at the Zentyal log directory /var/log/zentyal:
access.log error.log redis-server.log software.log zentyal.log
Fist of all, as a general tip to reproduce a suspected problem, it's generally a good idea to execute tail -f <logfile> and then hit the Enter key a few times, this way you can see the last log and clearly see the new log lines appearing in real time.
Example:
tail -f zentyal.log 2012/07/11 18:05:10 DEBUG> Sudo.pm:164 EBox::Sudo::_root - sudo commands: cat /var/run/clamav/clamd.pid 2012/07/11 18:05:10 DEBUG> Sudo.pm:164 EBox::Sudo::_root - sudo commands: status 'ebox.bind9' 2012/07/11 18:05:10 DEBUG> Sudo.pm:164 EBox::Sudo::_root - sudo commands: LANG=C /usr/sbin/ejabberdctl status 2012/07/11 18:05:10 DEBUG> Sudo.pm:164 EBox::Sudo::_root - sudo commands: status 'ebox.ntpd' 2012/07/11 18:05:10 DEBUG> Sudo.pm:164 EBox::Sudo::_root - sudo commands: status 'ebox.openvpn.server.vpnserver' 2012/07/11 18:05:10 DEBUG> Sudo.pm:164 EBox::Sudo::_root - sudo commands: status 'ebox.slapd' 2012/07/11 18:05:10 DEBUG> Sudo.pm:164 EBox::Sudo::_root - sudo commands: cat /var/run/heimdal-kdc.pid 2012/07/11 18:05:10 DEBUG> Sudo.pm:164 EBox::Sudo::_root - sudo commands: cat /var/run/kpasswdd.pid 2012/07/11 18:05:10 DEBUG> Sudo.pm:164 EBox::Sudo::_root - sudo commands: cat /var/run/apache2.pid 2012/07/11 18:06:14 DEBUG> Sudo.pm:164 EBox::Sudo::_root - sudo commands: status 'ebox.redis' 2012/07/11 18:06:15 INFO> GlobalImpl.pm:551 EBox::GlobalImpl::saveAllModules - Saving config and restarting services: network antivirus dns openvpn firewall 2012/07/11 18:06:15 INFO> Base.pm:229 EBox::Module::Base::save - Restarting service for module: network 2012/07/11 18:06:15 DEBUG> Sudo.pm:164 EBox::Sudo::_root - sudo commands: set -e /sbin/modprobe 8021q /sbin/vconfig set_name_type VLAN_PLUS_VID_NO_PAD 2012/07/11 18:06:15 DEBUG> Sudo.pm:164 EBox::Sudo::_root - sudo commands: cp /var/lib/zentyal/tmp//interfaces /etc/network/interfaces 2012/07/11 18:06:16 DEBUG> Sudo.pm:164 EBox::Sudo::_root - sudo commands: set -e /bin/mv /var/lib/zentyal/tmp/sjtCvBGY6U '/etc/ppp/pap-secrets'
access.log: Accesses to the Zentyal GUI, time, user, browser and requested URL, security forensics.
error.log: Apache error log, may be useful to detect invalid URL's or JavaScript errors (sometimes derived from deeper errors in the perl modules).
192.168.100.155 - - [19/Jun/2012:15:01:17 +0200] "POST /remoteservices/subscribe/ HTTP/1.1" 200 87 "-" "python-requests/0.8.1" "-" 192.168.100.155 - - [19/Jun/2012:15:08:15 +0200] "POST /remoteservices/subscribe/ HTTP/1.1" 200 87 "-" "python-requests/0.8.1" "-" 192.168.100.155 - - [19/Jun/2012:15:38:15 +0200] "POST /remoteservices/subscribe/ HTTP/1.1" 200 87 "-" "python-requests/0.8.1" "-"
redis-server.log: redis database consolidation logs
[22275] 12 Jul 18:02:52 * DB saved on disk [1946] 12 Jul 18:02:52 * Background saving terminated with success
software.log: Software packages managing information
2012-07-12 17:35:47> Setting up sshpass (1.05-1) ... 2012-07-12 17:35:47> Setting up zentyal-ebackup (2.3.5) ... 2012-07-12 17:35:49> Processing triggers for libc-bin ... 2012-07-12 17:35:49> ldconfig deferred processing now taking place 2012-07-12 17:35:50> Processing triggers for zentyal-core ... 2012-07-12 17:35:51> Zentyal apt-wrapper install finished
zentyal.log: Possibly the most important one to diagnose and correctly report an error, it contains general, debug and error information provided by the different Zentyal modules. Remember to enable the "debug = yes" key in /etc/zentyal/zentyal.conf, use the tail -f trick to try and reproduce the error live or try grep'ing around using the "ERROR>" string to gather error information. After changing this key, if you are trying to reproduce an error you get on the Zentyal web interface, you will need to restart it to reload the configuration:
# service zentyal apache restart *Restarting Zentyal module: apache [ OK ] # cat /var/log/zentyal/zentyal.log | grep ERROR\> -B 5 2012/07/12 17:19:14 DEBUG> Backup.pm:146 EBox::Backup::<u>ANON</u> - Dumping audit backup data 2012/07/12 17:19:14 DEBUG> Backup.pm:146 EBox::Backup::<u>ANON</u> - Dumping ca backup data 2012/07/12 17:19:14 DEBUG> Sudo.pm:165 EBox::Sudo::_root - ./make-backup (pid: 18788) - tar cf /var/lib/zentyal/conf//backup.s2sK0S/aux/ca.bak/srvCerts.tar /var/lib/zentyal/conf/ssl/ssl.pem /etc/apache2/ssl/ssl.pem 2012/07/12 17:19:14 DEBUG> Backup.pm:146 EBox::Backup::<u>ANON</u> - Dumping dns backup data 2012/07/12 17:19:14 DEBUG> Backup.pm:146 EBox::Backup::<u>ANON</u> - Dumping events backup data 2012/07/12 17:19:14 ERROR> Backup.pm:151 EBox::Backup::<u>ANON</u> - model LogWatcherFiltering_audit_sessions does not exist.
Generic daemon output files
This is a list of logfiles used by all Zentyal modules.
| Module | Path | logrotate upstream |
|---|---|---|
| zarafa | /var/log/zarafa/dagent.log | yes |
| zarafa | /var/log/zarafa/spooler.log | yes |
| zarafa | /var/log/zarafa/ical.log | yes |
| zarafa | /var/log/zarafa/monitor.log | yes |
| zarafa | /var/log/zarafa/gateway.log | yes |
| zarafa | /var/log/zarafa/indexer.log | yes |
| zarafa | /var/log/zarafa/server.log | yes |
| zarafa | /var/log/zarafa/audit.log | yes |
| samba | /var/log/samba/log.smbd | yes |
| samba | /var/log/samba/log.nmbd | yes |
| remoteservices | /home/ebox-remote-support/support.log | not rotated |
| ftp | /var/log/vsftpd.log | yes |
| core | /var/log/zentyal/access.log | zentyal-core |
| core | /var/log/zentyal/error.log | zentyal-core |
| core | /var/log/zentyal/zentyal.log | zentyal-core |
| core | /var/log/zentyal/redis-server.log | zentyal-core |
| core | /var/log/zentyal/runit/wild-services.log | not used atm |
| software | /var/log/zentyal/software.log | zentyal-software |
| openvpn | /var/log/quagga/ripd.log | yes |
| openvpn | /var/log/quagga/zebra.log | yes |
| openvpn | /var/log/zentyal/openvpn/*.log | zentyal-openvpn |
| usercorner | /var/log/zentyal-usercorner/access.log | zentyal-usercorner |
| usercorner | /var/log/zentyal-usercorner/error.log | zentyal-usercorner |
| usercorner | /var/log/zentyal-usercorner/zentyal.log | zentyal-usercorner |
| radius | /var/log/freeradius/radius.log | yes |
| /var/log/mail.log | yes | |
| ebackup | /var/log/zentyal/duplicity-debug.log | zentyal-ebackup |
| ebackup | /var/log/zentyal/duplicity-command.log | zentyal-ebackup |
| webserver | /var/log/apache2/*.log | yes |
| captiveportal | /var/log/zentyal-captiveportal/error.log | zentyal-captiveportal |
| captiveportal | /var/log/zentyal-captiveportal/access.log | zentyal-captiveportal |
| captiveportal | /var/log/zentyal-captiveportal/zentyal.log | zentyal-captiveportal |
| antivirus | /var/log/clamav/freshclam.log | yes |
| antivirus | /var/log/clamav/clamav.log | yes |
| squid | /var/log/dansguardian/access.log | |
| squid | /var/log/squid/access.log | |
| printers | /var/log/cups/*log | yes |
| jabber | /var/log/ejabberd/*.log | yes |
| ids | /var/log/snort/portscan.log | yes |
| ids | /var/log/snort/alert | yes |
| ids | /var/log/snort/portscan2.log | yes |
| asterisk | /var/log/asterisk/debug | yes |
| asterisk | /var/log/asterisk/messages | yes |
| asterisk | /var/log/asterisk/full | yes |
| asterisk | /var/log/asterisk/*_log | yes |
| webmail | /var/log/roundcube/sendmail | yes |
| webmail | /var/log/roundcube/errors | yes |
Useful commands and scripts
managing modules' status from the CLI
You can always stop, start or restart a given Zentyal module using the special /etc/init.d service zentyal.
Example:
# service zentyal firewall stop *Stopping Zentyal module: firewall [ OK ]
This may be useful is the GUI is temporarily unresponsive, or you have lost access to it.
slapcat
This command will display all your LDAP information as a file in your CLI, very useful for debugging and grep'ing if you master LDAP format.
# slapcat | head dn: dc=test,dc=org objectClass: organization objectClass: dcObject objectClass: top dc: test o: test structuralObjectClass: organization entryUUID: fdcde9be-4c09-1031-990f-bd2fd6edbea5 creatorsName: cn=zentyal,dc=test,dc=org createTimestamp: 20120616141913Z
Zentyal specific scripts
You will find some utility scripts under several directories named /usr/share/zentyal(*)
zentyal/ zentyal-firewall/ zentyal-objects/ zentyal-users/ zentyal-antivirus/ zentyal-jabber/ zentyal-openvpn/ zentyal-webserver/ zentyal-ca/ zentyal-network/ zentyal-services/ zentyal-dns/ zentyal-ntp/ zentyal-software/
Let's review some of them
zentyal/clean-conf : Delete the redis configuration associated with one of the zentyal modules.
Example:
./clean-conf openvpn
This will erase all the redis keys associated with that module. Useful when you reach an inconsistent configuration state during beta testing and you are not able to return to a safe point using the GUI, for example. Be careful, take into account that redis doesn't check relational integrity, so you may end with an incoherent DB.
zentyal/configuration-report : This script will create a configuration and associated files report and will leave it under /tmp. This report contains the essential system files needed to produce a diagnosis of an existing bug in your system.
zentyal/grep-redis: Inspect the redis configuration DB.
Example
./grep-redis section9
dns/conf/DomainTable/keys/dmn1/hostnames/keys/hstn1: {"hostname":"section9"}
dns/ro/DomainTable/keys/dmn1/hostnames/keys/hstn1: {"hostname":"section9"}
zentyal/initial-setup: Useful in combination with clean-conf, restores the default configuration keys of a given zentyal module.
zentyal/redisvi: Display the full redis key-value information, in an editable file format. Deep knowledge of the source code is required to safely tweak this database, handle with care.
zentyal/shell: Spawn the Zentyal shell, you can interactively use Zentyal's API using perl.
Example:
./shell
zentyal> instance network
$network
zentyal> $network->allIfaces()
$VAR1 = [
'eth0'
];
zentyal-software/unlock-software: Sometime you may have a problem with the software repositories of your Internet connection and several process may die before releasing the software lock, making it impossible to update
using the GUI. This script will kill the locking processes.
zentyal-ebackup/backup-tool: Performs a data backup according to the GUI configuration. Very useful when you want to test backup config, but don't want to wait until the scheduled backup time. Default is a full backup, passing a --incremental you can perform an incremental one.
