Bacula & Bareos: A Complete Backup Solution for the Home and Enterprise

Let’s talk about the Bacula & Bareos backup programs.

Preflight

Windows Platform
@!:{Before you deploy Bareos, you’ll need the following prerequisites installed and properly configured on the target server:
•    postgreSQL
•    PowerShell 2.0 or geater
This requires .Net Framework 2.0 or greater

Platform Info & Defaults Used
Item Description
OS Platform Windows Server 2003 R2 32-bit
Bareos Version 14.1.0 32-bit
PostgreSQL Version 9.3 32-bit
Bareos Service Password bareos
Installation Directory C:\Program Files\Bareos
Configuration Directory C:\Documents and Settings\All Users\Application Data\Bareos
Bareos Working Directory C:\ProgramData\Bareos

@!:{This article relies mostly on command line (cmd) user interaction

[divider]

About Bacula & Bareos

[divider]

What is Bacula?

A Client/Server-based backup program

In a nutshell:
Server component handles the backups, Client component sends the data to be backed up.

Simar to
Symantec Backup Exec
BackupPC
CrashPlan

Bacula website: http://blog.bacula.org/

What is Bareos?

Bareos is a fork of the Bacula project, so it boasts all of the same features as Bacula, as well as additional enhancements.

Bareos website: http://www.bareos.org/en/

For me, the most attractive feature of this fork is that the server component, the director, can now be installed on Windows.

Features

    Enterprise-level
Highly Scalable
Centrally managed
Secure
Multiplatform Client Component
Linux
OSX
Windows
Unix
Multiplatform Server Component
Bareos
Windows
Bacula
Linux
Unix
Free

How Does Bacula/Bareos Work?

wpid689-media_1418590354210.png

I yanked this verbatim from this website: http://www.netways.de/en/de/produkte/bacula/architecture_components/

Bacula Architecture & Components
Bacula is made up of several components that can be distributed to operate on several servers. Thus a central Bacula director demon can backup multiple servers, and in turn save data to multiple servers. Even a physically distributed backup strategy can be implemented easily and above all, centrally controlled.
Bacula’s individual components consist of:
Bacula director
Bacula director is the central program which controls and monitors all key tasks such as backup, restore, verify and archive. The director usually operates as a daemon or service.
Bacula console
Bacula console is the tool through which the administrator can communicate with the Bacula director. The console comes in 3 different versions, with the simplest and most popular being a text-based shell for Linux. The text-based console offers the greatest functional scope, while the GNOME and Windows GUIs provide fast backup and restore.
Bacula file demon
Bacula file demon is the actual backup client, installed on the machines to be backed up. It is operating system-specific and sends to-be-protected data with their attributes to the Bacula Director. Or in a recovery, it writes the data back onto disk. Bacula File Demon runs as a service on the servers to be backed up and is available for Unix / Linux and Windows.
Bacula storage
Bacula storage is responsible for storing and reading the saved data and their related attributes onto backup media. It runs as a service on the server which is connected to backup hardware (tapes or disks).
Bacula catalog
Bacula catalog is responsible for the indexing of all files and volumes. It allows the administrator to find and restore desired files quickly. All used volumes, saved files and exported jobs are saved in the catalog. This offers fast and efficient file restoration and management. Bacula currently supports MySQL, PostgreSQL and SQLite as catalog databases.

[divider]

Configuration Overview

Bacula/Bareos is pretty simple.

It is nothing but the sum configuration of its components. So, in essence, once you configure all of its ‘moving parts’, it will just work as intended.

Each component has its own config file
director
bacula-dir.conf
console
bconsole.conf
file daemon
bacula-fd.conf
storage
bacula-sd.conf
catalog
n/a

Each config file is formatted based on resources comprised of directives
Directives:
Specify values relative to the component
Are surrounded by curly braces {}

        @!:{For Bareos:
Configuration files are named as bareos-{}.conf instead of bacula-{}.conf

As a security measure, the various Bacula components must authorize themselves to each other
This is accomplished using password specification
example:
the Storage resource password in the bacula-dir.conf file must match the Director resource password in bacula-sd.conf

[divider]

Configuration Architecture

wpid697-media_1419381381377.png

By default, the Bacula/Bareos main configuration files are monolithic
Pros:
One config file
One location
Cons:
Difficult to manage
More prone to user error

Since Bacula 2.2.0 you can include the output of a command within a configuration file with the ”@|” syntax.
The same applies to Bareos
This allows us to break up the configs into separate, more manageable ‘child’ configuration files
How?
By parsing these ‘child’ configuration files
ForEach child config file
Read content
Incorporate into main config file

In our configuration, we will be using this special syntax to create a distributed configuration
That is …
The main config file will be built from child config files located in specified folders
Pros:
Compartmentalized
One folder per backup client
With each folder containing relevant client configs
Easier to manage
Easier to deploy new clients
This would involve:
Installing the backup client on the host
Creating a config folder for the client
Populating configuration files in the client folder
Cons:
Multiple configs
Multiple locations

The architecutre is as follows:
bacula-dir.conf
Any files in the “conf.dir” folder are considered part of this file
e.g.
On our Windows host, we have the following layout:
conf.dir [folder]
A ‘global’ folder contains default and global definitions for fileset, jobs, messages, pool, schedule, and storage directives
Client folders contain client-specific definitions for client, fileset, jobs, messages, pool, schedule, and storage directives
global [folder]
fileset.conf
jobs.conf
messages.conf
pools.conf
schedules.conf
storages.conf
NewJersey [folder]
njexch01 [folder]
configs {njexch01.fileset.conf; njexch01.jobs.conf; njexch01.pools.conf …}
NewYork [folder]
nyweb03 [folder]
configs {nyweb01.fileset.conf; nyweb01.jobs.conf; nyweb01.pools.conf …}
Pennsylvania [folder]
padb01 [folder]
configs {padb01.fileset.conf; padb01.jobs.conf; padb01.pools.conf …}
Florida [folder]
flexch01 [folder]
configs {flexch01.fileset.conf; flexch01.jobs.conf; flexch01.pools.conf …}
bacula-sd.conf
Any files in the “conf.sd” folder are considered part of this file
e.g.
On our Windows host, we have the following layout:
conf.sd [folder]
A ‘global’ folder contains default and global storage device definitions
Client folderscontains client-specific storage device definitions
global [folder]
default-device.conf
NewJersey [folder]
njexch01-device.conf
NewYork [folder]
nyweb03-device.conf
Pennsylvania [folder]
padb01-device.conf
Florida [folder]
flexch01-device.conf

 

File Paths

Windows Server 2003
[shell]C:\Documents and Settings\All Users\Application Data\Bareos\bareos-dir.conf
C:\Documents and Settings\All Users\Application Data\Bareos\bareos-dir.conf.readme.txt
C:\Documents and Settings\All Users\Application Data\Bareos\bareos-fd.conf
C:\Documents and Settings\All Users\Application Data\Bareos\bareos-sd.conf
C:\Documents and Settings\All Users\Application Data\Bareos\conf.dir
C:\Documents and Settings\All Users\Application Data\Bareos\conf.sd
C:\Documents and Settings\All Users\Application Data\Bareos\logs
C:\Documents and Settings\All Users\Application Data\Bareos\scripts
C:\Documents and Settings\All Users\Application Data\Bareos\tray-monitor.conf
C:\Documents and Settings\All Users\Application Data\Bareos\working
C:\ProgramData\Bareos\logs\bareos.log
C:\ProgramData\Bareos\logs\bareos-audit.log
C:\ProgramData\Bareos\working[/shell]

[divider]

Getting Started

Let’s get down to the guts shall we?

Installing Bacula on CentOS Linux 6.x

Installing Bareoes on Windows Server 2003 R2

Installing Bacula on CentOS Linux 6.x

@!:{this should work for the RedHat equivalent.

[divider]

Install MySQL and Bacula

[shell]yum install mysql-devel mysql-server
yum install bacula-storage-mysql bacula-docs
yum install bacula-director-mysql bacula-console
yum install bacula-client bacula-traymonitor[/shell]

Start and Configure MySQL for Bacula

[shell]service mysqld start
chkconfig mysqld on[/shell]

Change the MySQL root password if you have a fresh install of MySQL

[shell]mysqladmin -u root password ‘new-password'[/shell]

Creating the mysql database structure

wpid690-media_1418671835366.png

[shell]/usr/libexec/bacula/grant_mysql_privileges -u root -p
/usr/libexec/bacula/create_mysql_database -u root -p
/usr/libexec/bacula/make_mysql_tables -u root -p
/usr/libexec/bacula/grant_bacula_privileges -u root -p[/shell]

Installing from Source [Optional]

see:{http://wiki.bacula.org/doku.php?id=bacula_manual:installing_bacula
gls*install bacula from source
see:{Bacula Client For HP-UX@http://h30499.www3.hp.com/t5/System-Administration/Bacula-Client-For-HP-UX/td-p/5173539#.Ut7NwvQo5ok

Bacula: Post-Installation

Create the backup folder(s)
[shell]mkdir /backup
chown bacula /backup
chmod 766 /backup[/shell]

Set the MySQL password for user bacula
[shell]mysql -u root -p
–Enter Password:
WHERE user=’bacula’;
UPDATE mysql.user SET password=PASSWORD (‘somepassword’) WHERE user=’bacula’;
FLUSH PRIVILEGES;
quit[/shell]

Configure and Start the Bacula Services
[shell]chkconfig bacula-dir on
chkconfig bacula-fd on
chkconfig bacula-sd on
service bacula-dir start
service bacula-fd start
service bacula-sd start[/shell]

Update Firwall Rules (If Applicable)
Modify the iptables configuration file to allow traffic to the bacula-director
[shell]vi /etc/sysconfig/iptables[/shell]
#Allow TCP – Bacula Director
[shell]    -A INPUT -m state –state NEW -m tcp -p tcp –dport 9101 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 9102 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 9103-j ACCEPT
:q[/shell]
    Restart the firewall
[shell]service iptables restart[/shell]

[divider]

Installing Bareoes on Windows Server 2003 R2

@!:{Before you deploy Bareos, you’ll need the following prerequisites installed and properly configured on the target server:

  • postgreSQL
  • PowerShell Installed

Preflight Information
Item Description OS Platform Windows Server 2003 R2 32-bit Bareos Version 14.1.0 32-bit PostgreSQL Version 9.3 32-bit Bareos Service Password Being Used bareos Installation Directory C:\PostgreSQL\9.3 Data Directory C:\PostgreSQL\9.3\data

[divider]

Bareos Setup

wpid691-media_1418678844770.png

Installation
Installer Filename: winbareos-14.1.0.git.1406399071-32-bit-r786.1.exe
Launch the installer
Next
Agree to Licence • Next
Destination Folder: C:\Program Files\Bareos • Next
Type of install: Full • Next
Bareos Client Configuration
Client Name
bareos-fd
Director Name
bareos-dir
Password
bareos
Network Address
bareos.mydomain.local
Client Monitor Password
bareos
Bacula Compatibility
Unchecked

Post-Installation

After installation is complete:
Ensure sufficient permissions are in place for the Bareos Service Account
Bareos config paths
[shell]cacls “C:\Documents and Settings\All Users\Application Data\Bareos\scripts” /e /g bareos:f /t[/shell]
libbareoscats-postgresql.dll
[shell]copy “C:\Program Files\Bareos\libbareoscats-postgresql.dll” to %WINDIR%\System32[/shell]
Launch the PostgreSQL database creation scripts
Change Working Directory
[shell]cd “C:\Documents and Settings\All Users\Application Data\Bareos\scripts\”[/shell]
Create the bareos database
[shell]psql.exe -U <postgres username> -f postgresql-createdb.sql[/shell]
Grant Database Rights
[shell]psql.exe -U <postgres username> -f postgresql-grant.sql bareos[/shell]

This concludes installation on Windows Server 2003

[divider]

Configuring The Services

Great! You’ve installed Bacula/Bareos
Now it’s time to configure everything
The config files we are concerned with:
Windows
bareos-dir.conf
bareos-fd.conf
bareos-sd.conf
bat.conf
bconsole.conf
tray-monitor.conf

[divider]

Director Config [bacula-dir.conf] – Directives Explained

    Client
A pointer to the computer you want to back up
Schedule
Definition of when this job will run and the type of backup
Where type can be
Full
Incremental
Differential
Storage
A pointer to the backup device (tape drive or disk storage)
Catalogue
Details of the SQL database which stores the catalogue (index to contents of backup)
Pool
Collection of tapes or disk files which make up the storage
You may have multiple pools in different rotations
Messages
This is the Notification Engine
Can send messages to
Email
Log File
etc
Fileset
Path definitions for the backup selection
Answers the question: What are you backing up?
Allows inclusion/exclusion rules
Job
The Job directive can be considered the glue that binds all other directives in the Director config
The following specifications comprise this directive:
Client
Schedule
FileSet
Pool
The Pool of backup destinations
Where the destination is a backup device defined in a Storage directive

Director Config [bacula-dir.conf] – Sample

[md]@|”sh -c ‘powershell.exe -ExecutionPolicy ByPass -Command C:/Docume~1/AllUse~1/Applic~1/Bareos/scripts/Get-BareosConf.ps1′”

C:\Documents and Settings\All Users\Application Data\Bareos\bareos-sd.conf
#Includes
@|”sh -c ‘powershell.exe -ExecutionPolicy ByPass -Command C:/Docume~1/AllUse~1/Applic~1/Bareos/scripts/Get-BareosConf.sd.ps1′”[/md]

Storage Daemon Config [bacula-sd.conf] – Directives Explained

Storage
One storage record for general setup
Messages
This is the Notification Engine
Can send messages to
Director
Director
Defines the Director allowed to control the Storage Daemon
Name
Password
Device
Defines the storage device
Type
Tape
Disk

[divider]

Appendix

[divider]

Managing Backup Volumes

You might need to manage the backup voulmes for various reasons, e.g.
You need to clean up some unneeded volumes that are taking up valuable disk space

[divider]

Prune & Purge Multiple Volumes From The Command Line

wpid694-media_1419017741972.png

Bacula on Linux:
echo “use bacula;SELECT volumename
FROM Media,Pool,Storage
WHERE Media.PoolId=Pool.PoolId
AND lastwritten < DATE_SUB(CURDATE(), INTERVAL 1 DAY)
AND Pool.Name=’etejedadmc-full-pool’
AND Media.StorageId=Storage.StorageId
ORDER BY VolumeName ASC;
” | mysql -u root -p | tail -n+2 > MatchedVolumes.list
cat MatchedVolumes.list | xargs -n 1 -I % echo ‘prune volume=”%” yes’ | bconsole
cat MatchedVolumes.list | xargs -n 1 -I % echo ‘purge volume=”%” yes’ | bconsole
cat MatchedVolumes.list | xargs -n 1 -I % echo rm -f /mnt/backups/etejedabackups/regular/etejedadmc.ufn.local/%
cat MatchedVolumes.list | xargs -n 1 -I % echo rm -f /mnt/backups/etejedabackups/regular/etejedadmc.ufn.local/%
cat MatchedVolumes.list | xargs -n 1 -I % ls -lh /mnt/backups/etejedabackups/regular/etejedadmc.ufn.local/%
#verify purge status
cat MatchedVolumes.list | xargs -n 1 -I % echo list volume=% | bconsole |grep ‘^|’|tail -n +2|sed ‘s,|,,g’
#remove volumes
cat MatchedVolumes.list | xargs -n 1 -I % rm -f /mnt/backups/etejedabackups/regular/etejedadmc.ufn.local/%

Bareos on Windows:
echo prune volume=Default-Pool-Full-etejedadmc-fd-183-vol yes| bconsole.exe
echo prune volume=Default-Full-Full-etejedabak02-fd-23-vol yes | bconsole.exe
echo prune volume=Default-Pool-Full-etejedadmc-fd-182-vol yes | bconsole.exe
echo prune volume=Default-Pool-Full-etejedadmc-fd-40-vol yes | bconsole.exe
echo prune volume=Default-Pool-Full-etejedadmc-fd-44-vol yes | bconsole.exe
echo prune volume=Full-0001 yes | bconsole.exe
echo prune volume=Incremental-0002 yes | bconsole.exe
echo prune volume=Differential-0003 yes | bconsole.exe

echo purge volume=Default-Pool-Full-etejedadmc-fd-183-vol yes| bconsole.exe
echo purge volume=Default-Full-Full-etejedabak02-fd-23-vol yes | bconsole.exe
echo purge volume=Default-Pool-Full-etejedadmc-fd-182-vol yes | bconsole.exe
echo purge volume=Default-Pool-Full-etejedadmc-fd-40-vol yes | bconsole.exe
echo purge volume=Default-Pool-Full-etejedadmc-fd-44-vol yes | bconsole.exe
echo purge volume=Full-0001 yes | bconsole.exe
echo purge volume=Incremental-0002 yes | bconsole.exe
echo purge volume=Differential-0003 yes | bconsole.exe

How to change Maximum Volume size in Pool Definition

update pool from resource
followed by
update all volumes in pool
follow the prompts in both cases

[divider]

Interacting With Bacula/Bareos

[divider]

Commands Cheatsheet

Scripting Bconsole

Windows BAT: Bacula Admin Tool

wpid696-media_1419026004472.png

[divider]

Monitoring Bacula

This secions covers monitoring basics for the Bacula software

[divider]

Server-Side Key Performance Indicators (KPIs)

Solutions

    Nagios Monitoring:
http://exchange.nagios.org/directory/Plugins/Backup-and-Recovery/Bacula
Xymon Bacula Check Script
http://www.revpol.com/xymon_bacula_check_script

[divider]

Caveats

This section covers some warnings one should heed to avoid problems in the future.

[divider]

Bacula Output File

wpid695-media_1419023701702.png

If you use the default bacula-dir.conf or some variation of it, you will note that it logs all the Bacula output to a file.

To avoid that this file grows without limit, we recommend that you copy the file logrotate from the scripts/logrotate to /etc/logrotate.d/bacula.

This will cause the log file to be rotated once a month and kept for a maximum of five months.

You may want to edit this file to change the default log rotation preferences.

See:{http://www.bacula.org/en/dev-manual/main/main/Getting_Started_with_Bacula.html

[divider]

Scripts

Windows
Get-BareosConf.dir.ps1
$fso = New-Object -ComObject Scripting.FileSystemObject;
Get-ChildItem -Recurse “C:\Docume~1\AllUse~1\Applic~1\Bareos\conf.dir” `
| Where-Object {!$_.PSIsContainer} `
| ForEach-Object {
$conf=$fso.GetFile($_.FullName).ShortPath
‘@’ + “$conf”
}
Get-BareosConf.sd.ps1
$fso = New-Object -ComObject Scripting.FileSystemObject;
Get-ChildItem -Recurse “C:\Docume~1\AllUse~1\Applic~1\Bareos\conf.sd” `
| Where-Object {!$_.PSIsContainer} `
| ForEach-Object {
$conf=$fso.GetFile($_.FullName).ShortPath;’@’ + “$conf”
}

[divider]

Miscellaneous Tasks

[divider]

Reset Bacula Databases and Files

Linux
Backup Existing Database And Files
[shell]/usr/bin/mysqldump -aecqQ bacula > bacula_before_purge.sql[/shell]
Stop Bacula Services
[shell]service bacula-dir stop
service bacula-fd stop
service bacula-sd stop[/shell]
Drop MySQL Tables
[shell]mysql -u root -p -e “drop database bacula;”[/shell]
Recreate Database & Grant Permissions
/usr/libexec/bacula/create_mysql_database -u root -p
/usr/libexec/bacula/make_mysql_tables -u root -p
/usr/libexec/bacula/grant_mysql_privileges -u root -p
Stop Bacula Services
[shell]service bacula-dir start
service bacula-sd start
service bacula-fd start[/shell]
@!:{Assumes you are using MySQL for the Bacula Database

[divider]

Troubleshooting

    

Problem Scenarios
Problem Possible Cause Troubleshooting
The Bacula Director service fails to start or starts then quickly stops There is most likely an error in the configuration preventing proper function The first step in troubleshooting is to determine what the problem is
Try launching the Director interactively from command line (without the /service flag) and observing output, e.g.:
[shell]"C:\Program Files\Bareos\bareos-dir.exe" -c "C:\Documents and Settings\All Users\Application Data\Bareos\bareos-dir.conf"
[/shell] In my case, I had an error in the configuration, so I was presented with the following standard output: [diff] … bareos-dir: ERROR TERMINATION atlib/res.c:459
Config error: Could not find config Resource Full referenced on line 13 : Full Backup Pool = Full …
: line 13, col 43 of file C:\DOCUME~1\ALLUSE~1\APPLIC~1\Bareos\conf.dir\base\JOBS~1.CON[/diff]
[diff]Fatal error: Storage daemon didn't accept Device
"device-name" because: 3924 Device "device-name" not in SD Device resources or no matching Media Type[/diff]
Possibibilites include:
1. Mismatching Media Type definitions between client storage definition and bacula storage definition
2. Missing Media Type definitions in either the client storage definition, bacula storage definition, or both
3. Insufficient permissions on the backup destination folder (if Media Type is File)
4. Storage daemon not running
5. You updated parameters for a storage device or pertinant configuration, but did not restart the storage daemon to finalize the changes [attr style="width:300px"
[diff]Fatal error: Storage daemon didn't accept Device "device-name" command[/diff] An error in the Storage Daemon configuration Double-check your SD configuration
Common problems include:
1. Invalid or non-existing path specified in Device directive (applies to Disk backup types)
[diff]Error: Director's comm line to SD dropped.
… Fatal error: filed/dir_cmd.c:2208 Comm error with SD. bad response to Append Data. ERR=Input/output error[/diff]
An error in the Storage Daemon configuration Double-check your SD configuration
Common problems include:
1. Invalid or non-existing path specified in Device directive (applies to Disk backup types)

@!:{Again, make sure to restart the storage daemon if you make any changes to the storage daemon configuration!

Filesystem Permissions

wpid693-media_1418712327445.png

Sources

see:{Bacula volumes – running low on disk space@http://dan.langille.org/2012/10/04/bacula-volumes-running-low-on-disk-space/
see:{File storage: disk full, how to recover@http://bacula.10910.n7.nabble.com/File-storage-disk-full-how-to-recover-td40740.html
see:{pruneall@http://bacula.10910.n7.nabble.com/file/n40741/pruneall
see:{delpurged@http://bacula.10910.n7.nabble.com/file/n40741/delpurged
see:{gls*bacula delete failed jobs
see:{Purge Jobs @http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-mailing-lists-3/bacula-25/purge-jobs-103092/
see:{gls*bacula disk full recycle
see:{Bacula disk space management@http://www.bignose.ca/2013/09/19/bacula-disk-space-management/
see:{Run sql-command from bash-script?@http://stackoverflow.com/questions/20351573/run-sql-command-from-bash-script
see:{produce a separate sql@http://www.crumpeta.com/using_xargs_to_create_sql_statements
see:{Volume Status is Full instead of Purge@http://bacula.10910.n7.nabble.com/Volume-Status-is-Full-instead-of-Purge-td76522.html
see:{gls*bconsole list vol status purged
see:{Bacula Cheat Sheet@https://workaround.org/bacula-cheatsheet
see:{Reset Bacula database and files@http://blog.mansonthomas.com/2009/09/reset-bacula-database-and-files.html




Installing PostgreSQL on Windows Server 2003

This Article covers installation of PostgreSQL Version 9.3 32-bit on Windows Server 2003 32-bit OS

Preflight

Item Description OS Platform Windows Server 2003 R2 32-bit PostgreSQL Version 9.3 32-bit PostgreSQL Service Account Username :postgres
Password: postgres Installation Directory
C:\PostgreSQL\9.3 Data Directory C:\PostgreSQL\9.3\data

@!:{This article presents the steps from the command line (cmd) perspective.

Launch PostgreSQL Installation

wpid368-wpid-media_1418426929034.png

Installer Filename: postgresql-9.3.5-1-windows.exe
Launch the installer
Next • Data Directory: C:\PostgreSQL\9.3\data • Password: postgre • Port: 5432 • Locale: Default locale • Next …

Create The PostgreSQL Service User & Initialize Database Instance

wpid369-wpid-media_1418454067590.png

Create the PostgreSQL User Account
[shell]net user postgres /add[/shell]

Set the password for the User Account to ‘postgres’
[shell]net user postgres postgres[/shell]

Register the PostgreSQL Windows Service and set logon as the postgres user
[shell]pg_ctl register -N postgres -U postgres -P postgres -D "C:/PostgreSQL/9.3/data"[/shell]

Set appropriate permissions on the PostgreSQL data directory
[shell]cacls c:\PostgreSQL\9.3\data /e /g postgres:F[/shell]

Initialize the database instance, specifying the data directory from above
[shell]initdb -U postgres -A password -E utf8 -W -D "C:/PostgreSQL/9.3/data"[/shell]

Start the PostgreSQL Service
[shell]net start postgres[/shell]

This covers the installation and default configuration of PostgreSQL on Windows Server 2003 R2

Post-Installation Notes

Ensure the postgreSQL bin folder is in Path environmental variable

[divider]

Appendix

[divider]

Troubleshooting

Troubleshooting Scenarios
Error Possible Cause Possible Solution When you attempt starting the PostgreSQL service, you encounter an error similar to: "The postgres service on Local Computer started and then stopped. Some services stop automatically if they have no work to do, for example, the Performance Logs and Alerts service." The PostgreSQL Database Instance has not been initialized Initialize the database instance then try starting the service again
Additionally, you can verify that the PostgreSQL service account has adequate permissions on the data directory When you try executing any arbitrary PostgreSQL queries, you encounter an error similar to: "Execution of PostgreSQL by a user with administrative permissions is not permitted.
The server must be started under an unprivileged user ID to prevent
possible system security compromises. See the documentation for
more information on how to properly start the server.
You registered the PostgreSQL service with an administrative user De-register service
[shell]pg_ctl.exe unregister -N postgres
[/shell]
Register service as a non-administrative user account (e.g. postgres)
[shell]pg_ctl register -N postgres -U postgres -P postgres -D "C:/PostgreSQL/9.3/data"[/shell] You try to initialize the database instance but encount an error similar to: "initdb: directory "C:/PostgreSQL/9.3/data" exists but is not empty
If you want to create a new database system, either remove or empty
the directory "C:/PostgreSQL/9.3/data" or run initdb
with an argument other than "C:/PostgreSQL/9.3/data".
You are trying to initialize the database instance, but the data directory is not empty Delete the contents of the data directory and retry initializing the database instance Upon initializing the database instance, you encounter errors related to permissions, similar to:
fixing permissions on existing directory C:/PostgreSQL/9.3/data … initdb: could not change permissions of directory "C:/PostgreSQL/9.3/data": Permission denied
The PostgreSQL service account does not have adequate permissions on the data directory Grant full permissions on the data directory to the PostgreSQL service account, e.g.:
[shell]cacls c:\PostgreSQL\9.3\data /e /g postgres:F[/shell]




LAMP Stack with VirtualHosts On Centos 6.x

This article illustrates how to install the Apache Mysql PHP Stack on Centos 6.x.

Additionally, with this configuration, you can serve Multiple Domains using the Virtual Hosts Apache directive.

Install Apache

Invoke yum for installation of Apache
yum install -y httpd mod_ssl httpd-devel
@!:{httpd-devel libraries were included in order to have module compile capabilities, as well as being able to install modules from source

Enable autostart of the Apache service

chkconfig httpd on
Start the Apache service
service service httpd resart

Install PHP

Install PHP, et al

yum install -y php php-mysql php-common php-mbstring php-mcrypt php-devel php-xml php-pecl-memcache php-pspell php-snmp php-xmlrpc php-gd

Restart the Apache service

service httpd restart

Check DNS

Ensure there exists a DNS entry for the domain you want to use.

If this is a lab setup, or completely local, you can simply create a hosts entry for the domain, e.g.

vi /etc/hosts

[divider]

Virtual Hosts

The NameVirtualHost directive allows us to host multiple websites on a single web server.

Example:

You want to host mydomain1.com on your web server
You also want to host mydomain2.com on your web server

In order to accomplish this, you’ll need to:
– enable the NameVirtualHost directive
– create appropriate configuration files for the domains in question, e.g.:

/etc/httpd/conf.d/mydomain1.com.conf
/etc/httpd/conf.d/mydomain2.com.conf

For now, let’s configure just one domain, mydomain1.com:

[divider]

Create Vhosts Config Directories

Create a vhost config folder

mkdir -p /etc/httpd/vhost.d

Configure NameVirtualHost Directive

Add an include directive to the apache config file:

vim /etc/httpd/conf/httpd.conf
    Include vhost.d/*.conf

@!:{The above makes it so that any files ending in .conf under the folder vhost.d are included as part of the httpd.conf configuration
Notice that vhost.d is a relative path. The full path would be evaluated as ServerRoot/vhost.d, where ServerRoot is /etc/httpd (see the httpd.conf file for more information)

Comment out any Listen directives and add an include directive to a separate ports settings config file:

#Listen 12.34.56.78:80
#Listen 80
Include ports.conf

@!:{The above makes it so that the ports.conf file is included as part of the httpd.conf configuration
What this accomplishes is a separation of port specification from the main config file

Create a ports config file

vi /etc/httpd/ports.conf

With contents:

Listen $Port
NameVirtualHost $IPPUBLIC:$Port
NameVirtualHost $IPPRIVATE:$Port
NameVirtualHost *:$Port

Where $Port is the numeric value of the port number through which you want Apache to listen for traffic

#e.g.
NameVirtualHost 192.168.250.188:80
NameVirtualHost 127.0.0.1:80
NameVirtualHost *:80

Restart Apache

service httpd restart

Create The Config File for the Virtual Host/Domain

Create a config file for your domain

vim /etc/httpd/vhost.d/mydomain1.conf

 

   <VirtualHost *:80>

    ServerName mydomain1.com
    ServerAlias www.mydomain1.com
    DocumentRoot /var/www/vhosts/mydomain1.com
    <Directory /var/www/vhosts/mydomain1.com>
    Options Indexes FollowSymLinks MultiViews
    AllowOverride All
    </Directory>

    CustomLog /var/log/httpd/mydomain1.com-access.log combined
    ErrorLog /var/log/httpd/mydomain1.com-error.log

    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    LogLevel warn

    </VirtualHost>

Make sure your document root exists!

mkdir /var/www/vhosts/mydomain1.com
#–OR Try this One-liner–#
ls /var/www/vhosts/mydomain1.com 2> /dev/null || echo does not exist;echo creating folder;mkdir -p /var/www/vhosts/mydomain1.com && echo created folder!

[divider]

Modify Firewall

You’ll need to poke a hole in the firewall to allow communication to the Apache listening port (by default port 80):
Edit iptables config

vi /etc/sysconfig/iptables
A INPUT -m state –state NEW -m tcp -p tcp –dport 80 -j ACCEPT

Restart iptables

service iptables restart

[divider]

Troubleshooting

[divider]

Error – Could not find …

wpid326-media_1400872932471.jpg

1. Problem: When navigating to your domain via web browser, you receive an error similar to ‘could not find’

Q:{Is DNS setup correctly?
Check:

nslookup mydomain1.com

if error then ensure DNS record exists on your DNS server

if Windows, try the ipconfig /flushdns command

Q:{Is Firewall to blame?
Check:

telnet $yourdomain $port

e.g.

telnet mydomain1.com 80

if error then ensure Firewall port is open:

vi /etc/sysconfig/iptables
e.g. -A INPUT -m state –state NEW -m tcp -p tcp –dport 80 -j ACCEPT

Restart firewall:

service iptables restart

2. Test website access again
Hopefully Success!

3. Test PHP functionality:

vi /var/www/vhosts/domain.com/index.php

<?php
phpinfo();
?>
:wq

Test website access again

http://mydomain1.com/index.php

If you’ve made numerous changes, try restarting the Apache service again
service httpd restart
 
If all else fails, and if you have the option to do so, reboot the server
reboot

Error – requested URl was not found on this server

wpid327-media_1400873556606.jpg

In this case, I created the config file for the domain under vhosts.d, but had forgotten to give it a .conf file extension. doh!
Note how I used the watch command to ‘watch’ for changes to log files under /var/log/httpd.
This functions much like inotifywait for troubleshooting using log files.




Troubleshooting Ruby Applications

This article lists errors and solutions I’ve come across in my experience with Ruby applications.

Errors/Solutions

Common Errors/Possible Solutions
Error Description Possible Cause Possible Solution "did not find expected key while parsing a block mapping at line …" You may have a syntax error in a config/yaml file Try testing your yaml file with Ruby for any syntax errors.
If there is no output, then the syntax is OK. If errors found, start troubleshooting at the line number reported by the error message:
ruby -e "require 'yaml'; YAML.load_file('/somepath/myconfigfile.yml')"