27 May 2014 Karim Elatov

We decided to move away from Wordpress to Jekyll. There are many guides on the process and also a lot of reasons as well. Check out some posts on the process already:

Here are the steps to the process.

Prepare Github

Most of the steps are covered in GitHub Pages. First go and create your self an account on github.com:

github-account-creation

For the plan you can choose free:

github-plan-free

After you done with the creation it will take you to your github account:

github-account-created-login-page

After the account we have to create a repository with a special name. From Using Jekyll with Pages:

Using Jekyll

Every GitHub Page is run through Jekyll when you push content to a specially named branch within your repository. For User Pages, use the master branch in your username.github.io repository. For Project Pages, use the gh-pages branch in your project’s repository. Because a normal HTML site is also a valid Jekyll site, you don’t have to do anything special to keep your standard HTML files unchanged. Jekyll has thorough documentation that covers its features and usage. Simply start committing Jekyll formatted files and you’ll be using Jekyll in no time.

So let’s go ahead and create a repository with name of username.github.io. First select Create New Repository from the top:

create-new-repo-github

Then name it appropriately:

github-new-repo-name

After it’s created, you will see quick instructions on how to initialize the git repository:

github-initialization-instruct

After it’s done, go to settings of the repository and check the box that says Restrict editing to collabolators only:

github-restrict-editting

So let’s clone the repository and add a test home page:

[email protected]:~$git clone https://[email protected]/moxz1/moxz1.github.io.git
Initialized empty Git repository in /home/elatov/moxz1.github.io/.git/
Password:
warning: You appear to have cloned an empty repository.
[email protected]:~$cd moxz1.github.io/
[email protected]:~/moxz1.github.io$echo "Test Page" > index.html
[email protected]:~/moxz1.github.io$git add --all
[email protected]:~/moxz1.github.io$git commit -m "Initial Commit"
[master (root-commit) 197ba87] Initial Commit
 Committer: elatov <[email protected]>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:

    git config --global user.name "Your Name"
    git config --global user.email [email protected]

If the identity used for this commit is wrong, you can fix it with:

    git commit --amend --author='Your Name <[email protected]>'

 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 index.html
[email protected]:~/moxz1.github.io$git push origin master
Password:
Counting objects: 3, done.
Writing objects: 100% (3/3), 219 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To https://[email protected]/moxz1/moxz1.github.io.git
 * [new branch]      master -> master

Now if you look back on the settings page of the repository, it will let you know that your changes have been published:

site-is-published-github-settings

If you want you can update your git user configuration to reflect your username. And you can also use SSH keys instead regular password over https to do commits to the github repository. More information on how to configure the SSH keys are seen in Generating SSH Keys.

After sometime if you visit your user github page you will see the contents of the index.html file that you uploaded:

test-page-seen-from-github

Running Jekyll Locally

I was using a mac for my testing, so I decided to run jekyll locally. MacOSX comes with a ruby version but I didn’t want to mess with the system version. I already had macports setup on the mac (here is a post for the macports setup). Once you have macports, you can run the following to install the ruby version from macports:

sudo port install ruby19

Then create a symbolic link to ease the ruby usage:

sudo ln -s /opt/local/bin/ruby1.9 /opt/local/bin/ruby

Next install the ruby-gems

sudo port install rb-rubygems

Then fix the path for that as well:

sudo ln -s /opt/local/bin/gem1.9 /opt/local/bin/gem

When you install macports, it includes /opt/local/bin/ in the path prior to other locations. Make sure the gem and ruby versions by default are the macports one:

[email protected]:~$type -a gem
gem is /opt/local/bin/gem
gem is /usr/bin/gem
[email protected]:~$type -a ruby
ruby is /opt/local/bin/ruby
ruby is /usr/bin/ruby

Now let’s update the system gems:

[email protected]:~$sudo gem update --system
Updating rubygems-update
Fetching: rubygems-update-2.2.2.gem (100%)
Successfully installed rubygems-update-2.2.2
Installing RubyGems 2.2.2
RubyGems 2.2.2 installed
Installing ri documentation for rubygems-2.2.2

= 2.2.1 / 2014-01-06

Then install the rdoc gem before the jekyll install:

sudo gem install rdoc

Lastly go ahead and install jekyll

sudo gem install jekyll

Jekyll Structure

From Directory structure:

A basic Jekyll site usually looks something like this:

.
├── _config.yml
├── _drafts
|   ├── begin-with-the-crazy-ideas.textile
|   └── on-simplicity-in-technology.markdown
├── _includes
|   ├── footer.html
|   └── header.html
├── _layouts
|   ├── default.html
|   └── post.html
├── _posts
|   ├── 2007-10-29-why-every-programmer-should-play-nethack.textile
|   └── 2009-04-26-barcamp-boston-4-roundup.textile
├── _data
|   └── members.yml
├── _site
└── index.html

An overview of what each of these does:

FILE / DIRECTORY DESCRIPTION
_config.yml Stores configuration data. Many of these options can be specified from the command line executable but it’s easier to specify them here so you don’t have to remember them.
_drafts Drafts are unpublished posts. The format of these files is without a date: title.MARKUP. Learn how to work with drafts.
_includes These are the partials that can be mixed and matched by your layouts and posts to facilitate reuse. The liquid tag {\% include file.ext %} can be used to include the partial in _includes/file.ext.
_layouts These are the templates that wrap posts. Layouts are chosen on a post- by-post basis in the YAML front matter, which is described in the next section. The liquid tag `
<i class="fa fa-calendar"></i> 24 May 2014 <i class="fa fa-user"></i> <a href="http://virtuallyhyper.com">Karim Elatov</a> <div style="float: right;"><i class="fa fa-comments"></i> <a href="/2014/05/rhcsa-rhce-chapter-22-virtualization-with-kvm/#disqus_thread"></a></div>


  <div class="tag_box">
  <i class="fa fa-tags"></i> 
  
  


  
 
	<a href="/tags.html#KVM-ref">KVM <span>5</span></a>
 
	<a href="/tags.html#RHEL-ref">RHEL <span>3</span></a>
 
	<a href="/tags.html#libvirt-ref">libvirt <span>4</span></a>
 
	<a href="/tags.html#rhcsa_and_rhce-ref">rhcsa_and_rhce <span>13</span></a>

  



  </div>

<hr>
<h3 id="kvm">KVM</h3>

From the Virtualization Getting Started Guide:

What is KVM?

KVM (Kernel-based Virtual Machine) is a full virtualization solution for Linux on AMD64 and Intel 64 hardware that is built into the standard Red Hat Enterprise Linux 6 kernel. It can run multiple, unmodified Windows and Linux guest operating systems. The KVM hypervisor in Red Hat Enterprise Linux is managed with the libvirt API and tools built for libvirt (such as virt-manager and virsh). Virtual machines are executed and run as multi-threaded Linux processes controlled by these tools.

and from the Virtualization Tuning and Optimization Guide here is a quick overview:

kvm-overview.png

Installing The Hypervisor

From the Virtualization Host Configuration and Guest Installation Guide

###Configuring a Virtualization Host installation This section covers installing virtualization tools and virtualization packages as part of a fresh Red Hat Enterprise Linux installation.

Installing the virtualization package group

  1. Launch the Red Hat Enterprise Linux 6 installation program

    Start an interactive Red Hat Enterprise Linux 6 installation from the Red Hat Enterprise Linux Installation CD-ROM, DVD or PXE.

  2. Continue installation up to package selection

    Complete the other steps up to the package selection step.

    virt-host-select-rhel-install

    Select the Virtualization Host server role to install a platform for guest virtual machines.

Installing the virtualization packages with yum

To use virtualization on Red Hat Enterprise Linux you require at least the qemu-kvm and qemu-img packages. These packages provide the user-level KVM emulator and disk image manager on the host Red Hat Enterprise Linux system.

To install the qemu-kvm and qemu-img packages, run the following command:

# yum install qemu-kvm qemu-img

Several additional virtualization management packages are also available:

Recommended virtualization packages

  • python-virtinst - Provides the virt-install command for creating virtual machines.
  • libvirt - The libvirt package provides the server and host side libraries for interacting with hypervisors and host systems. The libvirt package provides the libvirtd daemon that handles the library calls, manages virtual machines and controls the hypervisor.
  • libvirt-python - The libvirt-python package contains a module that permits applications written in the Python programming language to use the interface supplied by the libvirt API.
  • virt-manager - virt-manager, also known as Virtual Machine Manager, provides a graphical tool for administering virtual machines. It uses libvirt-client library as the management API.
  • libvirt-client - The libvirt-client package provides the client-side APIs and libraries for accessing libvirt servers. The libvirt-client package includes the virsh command line tool to manage and control virtual machines and hypervisors from the command line or a special virtualization shell.

Install all of these recommended virtualization packages with the following command:

# yum install virt-manager libvirt libvirt-python python-virtinst libvirt-client

####Installing Virtualization package groups

The virtualization packages can also be installed from package groups. The following table describes the virtualization package groups and what they provide.

Virtualization Package Groups

virt-group-pkgs

To install a package group, run the yum groupinstall <groupname> command. For instance, to install the Virtualization Tools package group, run the yum groupinstall “Virtualization Tools” command.

Also from the Virtualization Host Configuration and Guest Installation Guide:

####Verifying virtualization extensions

Use this section to determine whether your system has the hardware virtualization extensions. Virtualization extensions (Intel VT-x or AMD-V) are required for full virtualization.

  1. Run the following command to verify the CPU virtualization extensions are available:

     $ grep -E 'svm|vmx' /proc/cpuinfo
    
  2. Analyze the output. The following output contains a vmx entry indicating an Intel processor with the Intel VT-x extension:

     flags   : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush 
         dts acpi mmx fxsr sse sse2 ss ht  tm syscall lm constant_tsc pni monitor ds_cpl
         vmx est tm2 cx16 xtpr lahf_lm
    

    The following output contains an svm entry indicating an AMD processor with the AMD-V extensions:

     flags   :  fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush
         mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni cx16
         lahf_lm cmp_legacy svm cr8legacy ts fid vid ttp tm stc
    

    If any output is received, the processor has the hardware virtualization extensions. However in some circumstances manufacturers disable the virtualization extensions in BIOS.

    The “flags:” output content may appear multiple times, once for each hyperthread, core or CPU on the system.

    The virtualization extensions may be disabled in the BIOS.

  3. Ensure KVM subsystem is loaded

    As an additional check, verify that the kvm modules are loaded in the kernel:

     # lsmod | grep kvm
    

    If the output includes kvm_intel or kvm_amd then the kvm hardware virtualization modules are loaded and your system meets requirements.

Install KVM Hypervisor Example

So let’s go ahead and install the necessary package to run KVM

[[email protected] ~]# yum install qemu-kvm qemu-img

My VM was running on VMware and I followed the instructions laid out here to enable nested hypervisor support. Jarret had a great blog on how to do it with ESXi 5.0. I was acutally on ESXi 5.5 and all that I had to do was add the following to the vmx of the VM:

~ # tail -1 /vmfs/volumes/datastore1/RHEL_3/RHEL_3.vmx
vhv.enable = "TRUE"

Then after installing the above packages I saw the kvm module loaded:

[[email protected] ~]# lsmod | grep kvm
kvm_intel              53484  0 
kvm                   316506  1 kvm_intel

and of course the CPU flags were there as well:

[[email protected] ~]# grep -E 'svm|vmx' /proc/cpuinfo
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts xtopology tsc_reliable nonstop_tsc aperfmperf unfair_spinlock pni pclmulqdq **vmx** ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx hypervisor lahf_lm ida arat xsaveopt pln pts dts tpr_shadow vnmi ept vpid
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts xtopology tsc_reliable nonstop_tsc aperfmperf unfair_spinlock pni pclmulqdq **vmx** ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx hypervisor lahf_lm ida arat xsaveopt pln pts dts tpr_shadow vnmi ept vpid

The libvirt daemon

From the Virtualization Administration Guide

The libvirt daemon provides an interface for managing virtual machines. You must have the libvirtd daemon installed and running on every remote host that needs managing.

$ ssh [email protected]
# chkconfig libvirtd on
# service libvirtd start

After libvirtd and SSH are configured you should be able to remotely access and manage your virtual machines. You should also be able to access your guests with VNC at this point.

Libvirt and libvirt tools

From the Virtualization Getting Started Guide:

The libvirt package is a hypervisor-independent virtualization API that is able to interact with the virtualization capabilities of a range of operating systems.

The libvirt package provides:

  • A common, generic, and stable layer to securely manage virtual machines on a host.
  • A common interface for managing local systems and networked hosts.
  • All of the APIs required to provision, create, modify, monitor, control, migrate, and stop virtual machines, but only if the hypervisor supports these operations. Although multiple hosts may be accessed with libvirt simultaneously, the APIs are limited to single node operations.

The libvirt package is designed as a building block for higher level management tools and applications, for example,virt-manager and the virsh command-line management tools. With the exception of migration capabilities, libvirt focuses on managing single hosts and provides APIs to enumerate, monitor and use the resources available on the managed node, including CPUs, memory, storage, networking and Non-Uniform Memory Access (NUMA) partitions. The management tools can be located on separate physical machines from the host using secure protocols.

Red Hat Enterprise Linux 6 supports libvirt and included libvirt-based tools as its default method for virtualization management (as in Red Hat Enterprise Virtualization Management).

The libvirt package is available as free software under the GNU Lesser General Public License. The libvirt project aims to provide a long term stable C API to virtualization management tools, running on top of varying hypervisor technologies. The libvirt package supports Xen on Red Hat Enterprise Linux 5, and it supports KVM on both Red Hat Enterprise Linux 5 and Red Hat Enterprise Linux 6.

  • virsh - The virsh command-line tool is built on the libvirt management API and operates as an alternative to the graphical virt-manager application. The virsh command can be used in read-only mode by unprivileged users or, with root access, full administration functionality. The virsh command is ideal for scripting virtualization administration.
  • virt-manager - virt-manager is a graphical desktop tool for managing virtual machines. It allows access to graphical guest consoles and can be used to perform virtualization administration, virtual machine creation, migration, and configuration tasks. The ability to view virtual machines, host statistics, device information and performance graphs is also provided. The local hypervisor and remote hypervisors can be managed through a single interface.
  • virt-install - virt-install is a command line tool to provision new virtual machines. It supports both text-based and graphical installations, using serial console, SDL, SPICE, or VNC client/server pair graphics. Installation media can be local, or exist remotely on an NFS, HTTP, or FTP server. The tool can also be configured to run unattended and kickstart the guest when installation is complete, allowing for easy automation of installation. This tool is installed as part of the python-virtinst package.

Virsh

From the Virtualization Administration Guide:

virsh is a command line interface tool for managing guest virtual machines and the hypervisor. The virsh command-line tool is built on the libvirt management API and operates as an alternative to the qemu-kvm command and the graphical virt-manager application. The virsh command can be used in read-only mode by unprivileged users or, with root access, full administration functionality. The virsh command is ideal for scripting virtualization administration.

virsh command quick reference

The following tables provide a quick reference for all virsh command line options.

####guest virtual machine management options

Command Shortcut (if available) Description
–help -h Prints basic help information.
–version [=short] -v Prints the libvirt library version
–version [=long] -V Prints the libvirt library version in addition to the options and the driver it was compiled in
–connect [URI] -c Connects to the specified URI as if done with the connect command instead of the default connection
–debug [level] -d Enables the debug messages to be logged that are at the debug level or higher. The Level range is 0-4 The default setting is 4. Virsh_Debug has level descriptors. (XREF)
–log [file] -l Writes logging details to the specified file.
–quiet -q Prevents informational messages from being displayed
–readonly -r Makes the initial connection as read-only, same as the –readonly option of the connect command.
–timing -t Outputs the elapsed time information for each command
–escape [string] -e Sets an alternative escape sequence for the console. By default ^] is used. Valid characters include alphabetic chatacters, @, [, ], \, ^, and _.

####guest virtual machine management commands

Command Description
list Lists all guest virtual machines.
dumpxml Outputs the XML configuration file for the guest virtual machine.
create Creates a guest virtual machine from an XML configuration file and starts the new guest virtual machine.
start Starts an inactive guest virtual machine.
destroy Forces a guest virtual machine to stop.
define Creates a guest virtual machine from an XML configuration file without starting the new guest virtual machine.
domid Displays the guest virtual machine’s ID.
domuuid Displays the guest virtual machine’s UUID.
dominfo Displays guest virtual machine information.
domname Displays the guest virtual machine’s name.
domstate Displays the state of a guest virtual machine.
quit Quits the interactive terminal.
reboot Reboots a guest virtual machine.
restore Restores a previously saved guest virtual machine stored in a file.
resume Resumes a paused guest virtual machine.
save Saves the present state of a guest virtual machine to a file.
shutdown Gracefully shuts down a guest virtual machine.
suspend Pauses a guest virtual machine.
undefine Deletes all files associated with a guest virtual machine.
migrate Migrates a guest virtual machine to another host physical machine.

The following virsh command options manage guest virtual machine and hypervisor resources:

Resource management options

Command Description
setmem Sets the allocated memory for a guest virtual machine. Refer to the virsh manpage for more details.
setmaxmem Sets maximum memory limit for the hypervisor. Refer to the virsh manpage for more details.
setvcpus Changes number of virtual CPUs assigned to a guest virtual machine. Refer to the virsh manpage for more details.
vcpuinfo Displays virtual CPU information about a guest virtual machine.
vcpupin Controls the virtual CPU affinity of a guest virtual machine.
domblkstat Displays block device statistics for a running guest virtual machine.
domifstat Displays network interface statistics for a running guest virtual machine.
attach-device Attach a device to a guest virtual machine, using a device definition in an XML file.
attach-disk Attaches a new disk device to a guest virtual machine.
attach-interface Attaches a new network interface to a guest virtual machine.
update-device Detaches a disk image from a guest virtual machine’s CD-ROM drive.
detach-device Detaches a device from a guest virtual machine, takes the same kind of XML descriptions as command attach-device.
detach-disk Detaches a disk device from a guest virtual machine.
detach-interface Detach a network interface from a guest virtual machine.

The virsh commands for managing and creating storage pools and volumes.

####Storage Pool options

Command Description
find-storage-pool-sources Returns the XML definition for all storage pools of a given type that could be found.
find-storage-pool-sources host port Returns data on all storage pools of a given type that could be found as XML. If the host physical machine and port are provided, this command can be run remotely.
pool-autostart Sets the storage pool to start at boot time.
pool-build The pool-build command builds a defined pool. This command can format disks and create partitions.
pool-create pool-create creates and starts a storage pool from the provided XML storage pool definition file.
pool-create-as name Creates and starts a storage pool from the provided parameters. If the –print-xml parameter is specified, the command prints the XML definition for the storage pool without creating the storage pool.
pool-define Creates a storage bool from an XML definition file but does not start the new storage pool.
pool-define-as name Creates but does not start, a storage pool from the provided parameters. If the –print-xml parameter is specified, the command prints the XML definition for the storage pool without creating the storage pool.
pool-destroy Permanently destroys a storage pool in libvirt. The raw data contained in the storage pool is not changed and can be recovered with the pool-create command.
pool-delete Destroys the storage resources used by a storage pool. This operation cannot be recovered. The storage pool still exists after this command but all data is deleted.
pool-dumpxml Prints the XML definition for a storage pool.
pool-edit Opens the XML definition file for a storage pool in the users default text editor.
pool-info Returns information about a storage pool.
pool-list Lists storage pools known to libvirt. By default, pool-list lists pools in use by active guest virtual machines. The –inactive parameter lists inactive pools and the –all parameter lists all pools.
pool-undefine Deletes the definition for an inactive storage pool.
pool-uuid Returns the UUID of the named pool.
pool-name Prints a storage pool’s name when provided the UUID of a storage pool.
pool-refresh Refreshes the list of volumes contained in a storage pool.
pool-start Starts a storage pool that is defined but inactive.

Volume options

Command Description
vol-create Create a volume from an XML file.
vol-create-from Create a volume using another volume as input.
vol-create-as Create a volume from a set of arguments.
vol-clone Clone a volume.
vol-delete Delete a volume.
vol-wipe Wipe a volume.
vol-dumpxml Show volume information in XML.
vol-info Show storage volume information.
vol-list List volumes.
vol-pool Returns the storage pool for a given volume key or path.
vol-path Returns the volume path for a given volume name or key.
vol-name Returns the volume name for a given volume key or path.
vol-key Returns the volume key for a given volume name or path.

Secret options

Command Description
secret-define Define or modify a secret from an XML file.
secret-dumpxml Show secret attributes in XML.
secret-set-value Set a secret value.
secret-get-value Output a secret value.
secret-undefine Undefine a secret.
secret-list List secrets.

Network filter options

Command Description
nwfilter-define Define or update a network filter from an XML file.
nwfilter-undefine Undefine a network filter.
nwfilter-dumpxml Show network filter information in XML.
nwfilter-list List network filters.
nwfilter-edit Edit XML configuration for a network filter.

This table contains virsh command options for snapshots:

Snapshot options

Command Description
snapshot-create Create a snapshot.
snapshot-current Get the current snapshot.
snapshot-delete Delete a domain snapshot.
snapshot-dumpxml Dump XML for a domain snapshot.
snapshot-list List snapshots for a domain.
snapshot-revert Revert a domain to a snapshot.

This table contains miscellaneous virsh commands:

Miscellaneous options

Command Description
version Displays the version of virsh.

Virsh Commands

From the same guide:

The commands in this section are generic because they are not specific to any domain.

help

$ virsh help [command|group] The help command can be used with or without options. When used without options, all commands are listed, one perline. When used with an option, it is grouped into categories, displaying the keyword for each group.

To display the commands that are only for a specific option, you need to give the keyword for that group as an option. For example:

$ virsh help pool
 Storage Pool (help keyword 'pool'):
	find-storage-pool-sources-as   find potential storage pool sources
	find-storage-pool-sources      discover potential storage pool sources
	pool-autostart                 autostart a pool
	pool-build                     build a pool
	pool-create-as                 create a pool from a set of args
	pool-create                    create a pool from an XML file
	pool-define-as                 define a pool from a set of args
	pool-define                    define (but don't start) a pool from an XML file
	pool-delete                    delete a pool
	pool-destroy                   destroy (stop) a pool
	pool-dumpxml                   pool information in XML
	pool-edit                      edit XML configuration for a storage pool
	pool-info                      storage pool information
	pool-list                      list pools
	pool-name                      convert a pool UUID to pool name
	pool-refresh                   refresh a pool
	pool-start                     start a (previously defined) inactive pool
	pool-undefine                  undefine an inactive pool
	pool-uuid                      convert a pool name to pool UUID

Using the same command with a command option, gives the help information on that one specific command. For example:

$virsh help vol-path
  NAME
	vol-path - returns the volume path for a given volume name or key

  SYNOPSIS
	vol-path <vol> [--pool <string>]

  OPTIONS
	[--vol] <string>  volume name or key
	--pool <string>  pool name or uuid

quit and exit

The quit command and the exit command will close the terminal. For example:

$virsh exit
$virsh quit

version

The version command displays the current libvirt version and displays information about where the build is from. For example:

$ virsh version
Compiled against library: libvirt 1.1.1
Using library: libvirt 1.1.1
Using API: QEMU 1.1.1
Running hypervisor: QEMU 1.5.3

connect

Connects to a hypervisor session. When the shell is first started this command runs automatically when the URI parameter is requested by the -c command. The URI specifies how to connect to the hypervisor. The most commonly used URIs are:

  • xen:///- connects to the local XEN hypervisor
  • qemu:///system - connects locally as root to the daemon supervising QEMU and KVM domains.
  • xen:///session - connects locally as a user to the user’s set of QEMU and KVM domains
  • lxc:/// - connects to a local Linux container

The command can be run as follows:

$virsh connect {name|URI}

Where {name} is the machine name (hostname) or URL (the output of the virsh uri command) of the hypervisor. To initiate a read-only connection, append the above command with –readonly. For more information on URIs refer to Remote URIs If you are unsure of the URI, the uri command will display it:

$ virsh uri
qemu:///session

Libvirt and virsh Example

So let’s go ahead and install libvirtd and ensure that it’s running:

[[email protected] ~]# yum install libvirt
[[email protected] ~]# chkconfig --list libvirtd
libvirtd       	0:off	1:off	2:off	3:on	4:on	5:on	6:off
[[email protected] ~]# service libvirtd status
libvirtd is stopped
[[email protected] ~]# service libvirtd start
Starting libvirtd daemon:                                  [  OK  ]
[[email protected] ~]# service libvirtd status
libvirtd (pid  2006) is running...

Now let’s install virsh and connect to libvirtd:

[[email protected] ~]# yum install libvirt-client

Now for using virsh:

[[email protected] ~]# virsh
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
	   'quit' to quit

virsh # connect qemu:///system

virsh # capabilities 
<capabilities>

  <host>
	<uuid>564d4d7b-76ee-e8f8-4f94-73a2120ac990</uuid>
	<cpu>
	  <arch>x86_64</arch>
	  <model>Westmere</model>
	  <vendor>Intel</vendor>
	  <topology sockets='2' cores='1' threads='1'/>
	  <feature name='rdtscp'/>
	  <feature name='hypervisor'/>
	  <feature name='avx'/>
	  <feature name='osxsave'/>
	  <feature name='xsave'/>
	  <feature name='vmx'/>
	  <feature name='pclmuldq'/>
	  <feature name='ss'/>
	  <feature name='ds'/>
	  <feature name='vme'/>
	</cpu>
	<power_management>
	  <suspend_disk/>
	</power_management>
	<migration_features>
	  <live/>
	  <uri_transports>
		<uri_transport>tcp</uri_transport>
	  </uri_transports>
	</migration_features>
	<topology>
	  <cells num='1'>
		<cell id='0'>
		  <cpus num='2'>
			<cpu id='0'/>
			<cpu id='1'/>
		  </cpus>
		</cell>
	  </cells>
	</topology>
	<secmodel>
	  <model>selinux</model>
	  <doi>0</doi>
	</secmodel>
	<secmodel>
	  <model>dac</model>
	  <doi>0</doi>
	</secmodel>
  </host>

  <guest>
	<os_type>hvm</os_type>
	<arch name='i686'>
	  <wordsize>32</wordsize>
	  <emulator>/usr/libexec/qemu-kvm</emulator>
	  <machine>rhel6.4.0</machine>
	  <machine canonical='rhel6.4.0'>pc</machine>
	  <machine>rhel6.3.0</machine>
	  <machine>rhel6.2.0</machine>
	  <machine>rhel6.1.0</machine>
	  <machine>rhel6.0.0</machine>
	  <machine>rhel5.5.0</machine>
	  <machine>rhel5.4.4</machine>
	  <machine>rhel5.4.0</machine>
	  <domain type='qemu'>
	  </domain>
	  <domain type='kvm'>
		<emulator>/usr/libexec/qemu-kvm</emulator>
	  </domain>
	</arch>
	<features>
	  <cpuselection/>
	  <deviceboot/>
	  <acpi default='on' toggle='yes'/>
	  <apic default='on' toggle='no'/>
	  <pae/>
	  <nonpae/>
	</features>
  </guest>

  <guest>
	<os_type>hvm</os_type>
	<arch name='x86_64'>
	  <wordsize>64</wordsize>
	  <emulator>/usr/libexec/qemu-kvm</emulator>
	  <machine>rhel6.4.0</machine>
	  <machine canonical='rhel6.4.0'>pc</machine>
	  <machine>rhel6.3.0</machine>
	  <machine>rhel6.2.0</machine>
	  <machine>rhel6.1.0</machine>
	  <machine>rhel6.0.0</machine>
	  <machine>rhel5.5.0</machine>
	  <machine>rhel5.4.4</machine>
	  <machine>rhel5.4.0</machine>
	  <domain type='qemu'>
	  </domain>
	  <domain type='kvm'>
		<emulator>/usr/libexec/qemu-kvm</emulator>
	  </domain>
	</arch>
	<features>
	  <cpuselection/>
	  <deviceboot/>
	  <acpi default='on' toggle='yes'/>
	  <apic default='on' toggle='no'/>
	</features>
  </guest>

</capabilities>


virsh # 

You can also run a command directly without going into the shell:

[[email protected] ~]#  virsh -c qemu:///system sysinfo | sed -n '/<bios>/,/<\/system>/p'
  <bios>
	<entry name='vendor'>Phoenix Technologies LTD</entry>
	<entry name='version'>6.00</entry>
	<entry name='date'>07/30/2013</entry>
	<entry name='release'>4.6</entry>
  </bios>
  <system>
	<entry name='manufacturer'>VMware, Inc.</entry>
	<entry name='product'>VMware Virtual Platform</entry>
	<entry name='version'>None</entry>
	<entry name='serial'>VMware-56 4d 4d 7b 76 ee </entry>
	<entry name='uuid'>564D4D7B-</entry>
	<entry name='sku'>Not Specified</entry>
	<entry name='family'>Not Specified</entry>
  </system>

Networking with libvirt

From the Virtualization Host Configuration and Guest Installation Guide:

Network Address Translation (NAT) with libvirt

One of the most common methods for sharing network connections is to use Network Address Translation (NAT) forwarding (also known as virtual networks).

Host configuration

Every standard libvirt installation provides NAT-based connectivity to virtual machines as the default virtual network. Verify that it is available with the virsh net-list --all command.

# virsh net-list --all
Name                 State      Autostart 
-----------------------------------------
default              active     yes

If it is missing, the example XML configuration file can be reloaded and activated:

# virsh net-define /usr/share/libvirt/networks/default.xml

The default network is defined from /usr/share/libvirt/networks/default.xml

Mark the default network to automatically start:

# virsh net-autostart default
Network default marked as autostarted

Start the default network:

# virsh net-start default
Network default started

Once the libvirt default network is running, you will see an isolated bridge device. This device does not have any physical interfaces added. The new device uses NAT and IP forwarding to connect to the physical network. Do not add new interfaces.

# brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.000000000000       yes

libvirt adds iptables rules which allow traffic to and from guest virtual machines attached to the virbr0 device in the INPUT, FORWARD, OUTPUT and POSTROUTING chains. libvirt then attempts to enable the ip_forward parameter. Some other applications may disable ip_forward, so the best option is to add the following to /etc/sysctl.conf.

net.ipv4.ip_forward = 1

Guest virtual machine configuration

Once the host configuration is complete, a guest virtual machine can be connected to the virtual network based on its name. To connect a guest to the ‘default’ virtual network, the following could be used in the XML configuration file (such as /etc/libvirtd/qemu/myguest.xml) for the guest:

<interface type='network'>
   <source network='default'/>
</interface>

Bridged networking with libvirt

Bridged networking (also known as physical device sharing) is used to dedicate a physical device to a virtual machine. Bridging is often used for more advanced setups and on servers with multiple network interfaces.

To create a bridge (br0) based on the eth0 interface, execute the following command on the host:

# virsh iface-bridge eth0 br0

Important

NetworkManager does not support bridging. NetworkManager must be disabled to use networking with the network scripts (located in the /etc/sysconfig/network-scripts/ directory).

# chkconfig NetworkManager off
# chkconfig network on
# service NetworkManager stop
# service network start

If you do not want to disable NetworkManager entirely, add “NM_CONTROLLED=no” to the ifcfg-* network script being used for the bridge.

VM Images and Installation Locations

From the Virtualization_Administration_Guide:

Use a central location for virtual machine installations and images. Virtual machine images should be stored under /var/lib/libvirt/images/. If you are using a different directory for your virtual machine images make sure you add the directory to your SELinux policy and relabel it before starting the installation. Use of shareable, network storage in a central location is highly recommended.

and from the Virtualization_Host_Configuration_and_Guest_Installation_Guide:

For ISO image files and guest storage images, the recommended location to use is /var/lib/libvirt/images/. Any other location may require additional configuration by SELinux.

Libvirt Networking and VM Image Location Example

Let’s make sure the default network is automatically configured:

[[email protected] ~]# virsh -c qemu:///system net-list
Name                 State      Autostart     Persistent
--------------------------------------------------
default              active     yes           yes

Now let’s make sure it’s assigned to a bridge:

[[email protected] ~]# virsh -c qemu:///system net-info default
Name            default
UUID            8405fc50-ead0-4afb-85bc-9baf521c2e27
Active:         yes
Persistent:     yes
Autostart:      yes
Bridge:         virbr0

Then install bridge-utils and check the bridge:

[[email protected] ~]# yum install bridge-utils
[[email protected] ~]# brctl show
bridge name	bridge id		STP enabled	interfaces
virbr0		8000.525400e08ccf	yes		virbr0-nic

Here are the firewall rules that were added by the libvirt package:

[[email protected] ~]# iptables -L -n -v
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
	0     0 ACCEPT     udp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           udp dpt:53 
	0     0 ACCEPT     tcp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           tcp dpt:53 
	0     0 ACCEPT     udp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           udp dpt:67 
	0     0 ACCEPT     tcp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           tcp dpt:67 
 4126  298K ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
	0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0           
	0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
	6   360 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22 
  127 34552 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
	0     0 ACCEPT     all  --  *      virbr0  0.0.0.0/0            192.168.122.0/24    state RELATED,ESTABLISHED 
	0     0 ACCEPT     all  --  virbr0 *       192.168.122.0/24     0.0.0.0/0           
	0     0 ACCEPT     all  --  virbr0 virbr0  0.0.0.0/0            0.0.0.0/0           
	0     0 REJECT     all  --  *      virbr0  0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 
	0     0 REJECT     all  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 
	0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain OUTPUT (policy ACCEPT 1593 packets, 241K bytes)
 pkts bytes target     prot opt in     out     source               destination         

And here is the nat table:

[[email protected] ~]# iptables -L -n -v -t nat
Chain PREROUTING (policy ACCEPT 73 packets, 14236 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 12 packets, 699 bytes)
 pkts bytes target     prot opt in     out     source               destination         
	0     0 MASQUERADE  tcp  --  *      *       192.168.122.0/24    !192.168.122.0/24    masq ports: 1024-65535 
	0     0 MASQUERADE  udp  --  *      *       192.168.122.0/24    !192.168.122.0/24    masq ports: 1024-65535 
	0     0 MASQUERADE  all  --  *      *       192.168.122.0/24    !192.168.122.0/24    

Chain OUTPUT (policy ACCEPT 12 packets, 699 bytes)
 pkts bytes target     prot opt in     out     source               destination         

I was okay with the NAT approach for now. I also wanted to make sure the ip_forward flag was applied:

[[email protected] ~]# cat /proc/sys/net/ipv4/ip_forward 
0
[[email protected] ~]# grep ip_forward /etc/sysctl.conf 
net.ipv4.ip_forward = 0

and it actually wasn’t for some reason, so I set the above value to 1 and then ran the following to apply the change:

[[email protected] ~]# sysctl -p
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296

Now checking the setting:

[[email protected] ~]# cat /proc/sys/net/ipv4/ip_forward 
1

That’s better.

It sounds like we need to use the /var/lib/libvirt/images directory for storing ISOs and virtual disks. Let’s make sure it exists and has approproate selinux context:

[[email protected] ~]# ls -dZ /var/lib/libvirt/images
drwx--x--x. root root system_u:object_r:virt_image_t:s0 /var/lib/libvirt/images

That looks good. I copied a centos ISO in there which I will use for my VM install later:

[[email protected] ~]# rsync -avzP 192.168.1.101:/data/isos/centos/CentOS-6.5-x86_64-minimal.iso /var/lib/libvirt/images/.
Password: 
receiving incremental file list

sent 11 bytes  received 62 bytes  29.20 bytes/sec
total size is 417333248  speedup is 5716893.81

VM Graphical framebuffers

From the Virtualization_Administration_Guide:

A graphics device allows for graphical interaction with the guest virtual machine OS. A guest virtual machine will typically have either a framebuffer or a text console configured to allow interaction with the admin.

To specify the graphical framebuffer devices configuration settings, use a management tool to make the following changes to the domain XML:

<devices>
   <graphics type='sdl' display=':0.0'/>
   <graphics type='vnc' port='5904'>
      <listen type='address' address='1.2.3.4'/>
   </graphics>
  <graphics type='rdp' autoport='yes' multiUser='yes' />
  <graphics type='desktop' fullscreen='yes'/>
  <graphics type='spice'>
    <listen type='network' network='rednet'/>
 </graphics>
 </devices>

The graphics element has a mandatory type attribute which takes the value sdl, vnc, rdp or desktop as explained below:

Graphical framebuffer elements

Parameter Description
sdl This displays a window on the host physical machine desktop, it can take 3 optional arguments: a display attribute for the display to use, an xauth attribute for the authentication identifier, and an optional fullscreen attribute accepting values yes or no
vnc Starts a VNC server. The port attribute specifies the TCP port number (with -1 as legacy syntax indicating that it should be auto-allocated). The autoport attribute is the new preferred syntax for indicating autoallocation of the TCP port to use. The listen attribute is an IP address for the server to listen on. The passwd attribute provides a VNC password in clear text. The keymap attribute specifies the keymap to use. It is possible to set a limit on the validity of the password be giving an timestamp passwdValidTo=’2010-04-09T15:51:00’ assumed to be in UTC. The connected attribute allows control of connected client during password changes. VNC accepts keep value only and note that it may not be supported by all hypervisors. Rather than using listen/port, QEMU supports a socket attribute for listening on a unix domain socket path.
spice Starts a SPICE server. The port attribute specifies the TCP port number (with -1 as legacy syntax indicating that it should be auto-allocated), while tlsPort gives an alternative secure port number. The autoport attribute is the new preferred syntax for indicating auto-allocation of both port numbers. The listen attribute is an IP address for the server to listen on. The passwd attribute provides a SPICE password in clear text. The keymap attribute specifies the keymap to use. It is possible to set a limit on the validity of the password be giving an timestamp passwdValidTo=’2010-04-09T15:51:00’ assumed to be in UTC. The connected attribute allows control of connected client during password changes. SPICE accepts keep to keep client connected, disconnect to disconnect client and fail to fail changing password. Note it is not be supported by all hypervisors. The defaultMode attribute sets the default channel security policy, valid values are secure, insecure and the default any (which is secure if possible, but falls back to insecure rather than erroring out if no secure path is available).

Prepare firewall for VNC Example

I will just use VNC to connect to the VM’s console. If I am going to do that, I will need to open up the firewall to allow 5900-5910 at least for testing. We actually covered this in Chapter 13. Here is the command to do that:

[[email protected] ~]# iptables -I INPUT 8 -m state --state NEW -m tcp -p tcp --dport 5900:5910 -j ACCEPT
[[email protected] ~]# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]

If you want to use the SPICE protocol, check out my previous post here.

Creating guests with virt-install

From the Virtualization_Host_Configuration_and_Guest_Installation_Guide:

You can use the virt-install command to create guest virtual machines from the command line. virt-install is used either interactively or as part of a script to automate the creation of virtual machines. Using virt-install with Kickstart files allows for unattended installation of virtual machines.

The virt-install tool provides a number of options that can be passed on the command line. To see a complete list of options run the following command:

# virt-install --help

Note that you need root privileges in order for virt-install commands to complete successfully. The virt-install man page also documents each command option and important variables.

qemu-img is a related command which may be used before virt-install to configure storage options.

An important option is the –graphics option which allows graphical installation of a virtual machine.

Using virt-install to install a Red Hat Enterprise Linux 5 guest virtual machine

This example creates a Red Hat Enterprise Linux 5 guest:

virt-install \
   --name=guest1-rhel5-64 \
   --file=/var/lib/libvirt/images/guest1-rhel5-64.dsk \
   --file-size=8 \
   --nonsparse --graphics spice \
   --vcpus=2 --ram=2048 \
   --location=http://example1.com/installation_tree/RHEL5.6-Server-x86_64/os \
   --network bridge=br0 \
   --os-type=linux \
   --os-variant=rhel5.4

Ensure that you select the correct os-type for your operating system when running this command.

Creating a VM with virt-install Example

First let’s install the utility:

[[email protected] ~]# yum install python-virtinst

Here is a command that will create the VM:

[[email protected] ~]# virt-install --name=centos --ram=512 --vcpus=1 --cdrom=/var/lib/libvirt/images/CentOS-6.5-x86_64-minimal.iso --os-type=linux --os-variant=rhel6 --disk=/var/lib/libvirt/images/centos.img,size=5 --network network=default --graphics=vnc,listen=0.0.0.0,keymap=en-us --noautoconsole

Starting install...
Allocating 'centos.img'                                                                                         | 5.0 GB     00:00     
Creating domain...                                                                                              |    0 B     00:00     
Domain installation still in progress. You can reconnect to 
the console to complete the installation process.

Now make sure the VM is running:

[[email protected] ~]# virsh list
 Id    Name                           State
----------------------------------------------------
 1     centos                         running

Now find the vncdisplay assigned to the VM:

[[email protected] ~]# virsh vncdisplay 1
127.0.0.1:0

[[email protected] ~]# netstat -ant | grep 590
tcp        0      0 0.0.0.0:5900                0.0.0.0:*                   LISTEN 

Looks like it’s just 5900. Now let’s install virt-viewer:

[[email protected] ~]# yum install virt-viewer xauth xorg-x11-fonts-Type1

and reconnect to our host with X-forwarding:

[email protected]:~$ssh -X rhel3 -l root
[email protected]'s password: 
Last login: Sat May 24 18:04:13 2014 from fed.local.com
/usr/bin/xauth:  creating new authority file /root/.Xauthority
[[email protected] ~]# virt-viewer 1

and you should see the following:

virt-viewer-launched-x-forward

From there we can just finish the install. Let’s stop there, shutdown the VM, and delete it:

[[email protected] ~]# virsh
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
	   'quit' to quit

virsh # list
 Id    Name                           State
----------------------------------------------------
 1     centos                         running

virsh # destroy 1
Domain 1 destroyed

virsh # list --all
 Id    Name                           State
----------------------------------------------------
 -     centos                         shut off

virsh # undefine centos
Domain centos has been undefined

Lastly remove the disk image file that was created for the VM:

[[email protected] ~]# rm /var/lib/libvirt/images/centos.img 
rm: remove regular file '/var/lib/libvirt/images/centos.img'? y

Since I started the VNC server for the VM on 0.0.0.0 (remote and not just local), I could connect to the VM from a remote machine using a regular vnc client:

[email protected]:~$vncviewer rhel3:0

TigerVNC Viewer 64-bit v1.3.0 (20140319)
Built on Mar 19 2014 at 17:09:18
Copyright (C) 1999-2011 TigerVNC Team and many others (see README.txt)
See http://www.tigervnc.org for information on TigerVNC.

Sat May 24 18:24:31 2014
 CConn:       connected to host rhel3 port 5900
 CConnection: Server supports RFB protocol version 3.8
 CConnection: Using RFB protocol version 3.8
 PlatformPixelBuffer: Using default colormap and visual, TrueColor, depth 24.
 CConn:       Using pixel format depth 24 (32bpp) little-endian rgb888
 CConn:       Using Tight encoding
 Viewport:    Unexpected release of FLTK key code 65293 (0xff0d)

and you will again see the same window:

vncviewer-connected-to-kvm-vm

If you are always able to SSH forward then might as well do that for security reasons, and then you can close the 5900 ports in the firewall. This was just for demostration purposes.

Creating guests with virt-manager

From the Virtualization_Host_Configuration_and_Guest_Installation_Guide:

virt-manager, also known as Virtual Machine Manager, is a graphical tool for creating and managing guest virtual machines.

Creating a guest virtual machine with virt-manager

  1. Open virt-manager

    Start virt-manager. Launch the Virtual Machine Manager application from the Applications menu and System Tools submenu. Alternatively, run the virt-manager command as root.

  2. Optional: Open a remote hypervisor

    Select the hypervisor and click the Connect button to connect to the remote hypervisor.

  3. Create a new virtual machine

    The virt-manager window allows you to create a new virtual machine. Click the Create a new virtual machine button to open the New VM wizard.

    virt-man-p1

    The New VM wizard breaks down the virtual machine creation process into five steps:

    1. Naming the guest virtual machine and choosing the installation type
    2. Locating and configuring the installation media
    3. Configuring memory and CPU options
    4. Configuring the virtual machine’s storage
    5. Configuring networking, architecture, and other hardware settings

    Ensure that virt-manager can access the installation media (whether locally or over the network) before you continue.

  4. Specify name and installation type

    The guest virtual machine creation process starts with the selection of a name and installation type. Virtual machine names can have underscores (_), periods (.), and hyphens (-).

    virt-man-p2

    Type in a virtual machine name and choose an installation type:

    • Local install media (ISO image or CDROM) - This method uses a CD-ROM, DVD, or image of an installation disk (for example, .iso).
    • Network Install (HTTP, FTP, or NFS) - This method involves the use of a mirrored Red Hat Enterprise Linux or Fedora installation tree to install a guest. The installation tree must be accessible through either HTTP, FTP, or NFS.
    • Network Boot (PXE) - This method uses a Preboot eXecution Environment (PXE) server to install the guest virtual machine. To install via network boot, the guest must have a routable IP address or shared network device.
    • Import existing disk image - This method allows you to create a new guest virtual machine and import a disk image (containing a pre-installed, bootable operating system) to it.

    Click Forward to continue.

  5. Configure installation

    Next, configure the OS type and Version of the installation. Ensure that you select the appropriate OS type for your virtual machine. Depending on the method of installation, provide the install URL or existing storage path.

    virt-man-p3

    virt-man-p4

  6. Configure CPU and memory

    The next step involves configuring the number of CPUs and amount of memory to allocate to the virtual machine. The wizard shows the number of CPUs and amount of memory you can allocate; configure these settings and click Forward.

    virt-man-p5

  7. Configure storage

    Assign storage to the guest virtual machine.

    virt-man-p5-5.png

    If you chose to import an existing disk image during the first step, virt-manager will skip this step.

    Assign sufficient space for your virtual machine and any applications it requires, then click Forward to continue.

  8. Final configuration

    Verify the settings of the virtual machine and click Finish when you are satisfied; doing so will create the virtual machine with default networking settings, virtualization type, and architecture.

    virt-man-p6

    If you prefer to further configure the virtual machine’s hardware first, check the Customize configuration before install box first before clicking Finish. Doing so will open another wizard that will allow you to add, remove, and configure the virtual machine’s hardware settings.

    After configuring the virtual machine’s hardware, click Apply. virt-manager will then create the virtual machine with your specified hardware settings.

Creating VM with virt-manager Example

First let’s install the utlility:

[[email protected] ~]# yum install virt-manager

Then launch the application if you are already using SSH X-Forwaring, if not, relogin with ssh enabling X-Forwaring (ssh -X).

[[email protected] ~]# virt-manager 
process 11006: D-Bus library appears to be incorrectly set up; failed to read machine uuid: Failed to open "/var/lib/dbus/machine-id": No such file or directory
See the manual page for dbus-uuidgen to correct this issue.
  D-Bus not built with -rdynamic so unable to print a backtrace
Aborted

Initially I ran into issue launching that. It turns out this is a know issue and there was a bug filed on it. Here is link to the bug, to fix the issue, just run the following:

[[email protected] ~]# dbus-uuidgen > /var/lib/dbus/machine-id

or the following:

[[email protected] ~]# service messagebus start
Starting system message bus:                               [  OK  ]

After that, it started up without issues:

[[email protected] ~]# virt-manager 
[[email protected] ~]# 

And I saw the following window:

virt-man-launched

We can see that is already connected to the qemu system, which is great. Now let’s click “New” and you should see the following pop up:

virt-man-new-vm-p1

Name the VM, select Local install media, and click Forward. On the next page, choose to point to an ISO and upon clicking on that you will see the default volume (/var/lib/libvirt/images). From there you can select the ISO:

virt-man-select-centos-iso

Then select the OS Type and Version. After done, it should look like this:

virt-man-os-selected

Click Forward, select RAM and CPU settings, it should look like this:

virt-man-ram-cpu-settings

Click Forward and select the disk size:

virt-man-vm-disk-size

Click Forward and you should see the final page:

virt-man-final-page

Upon clicking Finish you will see the console of the VM:

virt-man-console-open-after-install

You will also see the VM in the Virt-Manager’s Inventory:

virt-man-centos-running

If you proceed with the VM install, here is how the disk setup page will look like:

virt-man-centos-install-disk-page

After the install is finished, you will see the OS booted up:

virt-man-os-booted

You can also check on the KVM host it self to see all the parameters passed to the qemu-kvm binary when it starts up the VM, by checking out the VM specific logs that libvirt generates:

[[email protected] ~]# tail -3 /var/log/libvirt/qemu/centos.log 
2014-05-25 01:12:47.827+0000: starting up
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name centos -S -M rhel6.4.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -uuid 6eeb6219-97d4-7209-b422-b622494a12aa -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/centos.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/centos.img,if=none,id=drive-virtio-disk0,format=raw,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=22,id=hostnet0,vhost=on,vhostfd=23 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:69:0c:ab,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
char device redirected to /dev/pts/3

To find out the IP of the VM you can do a couple of things. First you can find the MAC address of the VM, with the following command:

[[email protected] ~]# virsh list
 Id    Name                           State
----------------------------------------------------
 4     centos                         running

[[email protected] ~]# virsh domiflist 4
Interface  Type       Source     Model       MAC
-------------------------------------------------------
vnet0      network    default    virtio      52:54:00:69:0c:ab

libvirt uses dnsmasq for it’s DHCP services, and all the leases are under /var/lib/libvirt/dnsmasq/default.leases. You can check the process table to see how dnsmasq was started:

[[email protected] ~]# ps -eaf | grep dnsma
nobody    2096     1  0 May24 ?        00:00:00 /usr/sbin/dnsmasq --strict-order --local=// --domain-needed --pid-file=/var/run/libvirt/network/default.pid --conf-file= --except-interface lo --bind-interfaces --listen-address 192.168.122.1 --dhcp-range 192.168.122.2,192.168.122.254 --dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases --dhcp-lease-max=253 --dhcp-no-override --dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile --addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts

And /var/log/messages, will show all the DHCP leases as well:

[[email protected] log]# tail -4 messages
May 25 11:18:16 rhel3 dnsmasq-dhcp[2096]: DHCPREQUEST(virbr0) 192.168.122.85 52:54:00:69:0c:ab 
May 25 11:18:16 rhel3 dnsmasq-dhcp[2096]: DHCPACK(virbr0) 192.168.122.85 52:54:00:69:0c:ab 
May 25 11:40:38 rhel3 dnsmasq-dhcp[2096]: DHCPREQUEST(virbr0) 192.168.122.85 52:54:00:69:0c:ab 
May 25 11:40:38 rhel3 dnsmasq-dhcp[2096]: DHCPACK(virbr0) 192.168.122.85 52:54:00:69:0c:ab 

But we can just check out the file to see what our current DHCP leases are:

[[email protected] ~]# cat /var/lib/libvirt/dnsmasq/default.leases
1400985508 52:54:00:69:0c:ab 192.168.122.85 * *

We only one have now, but if you had multiple address, you could run the following little bash command to track down your VM:

[[email protected] ~]# for mac in `virsh domiflist centos |grep -o -E "([0-9a-f]{2}:){5}([0-9a-f]{2})"`; do grep $mac /var/lib/libvirt/dnsmasq/default.leases; done | awk '{print $3}'
192.168.122.85
[[email protected] ~]# 

We can confirm by checking out the console:

ip-command-within-the-vm-virt-man

Or logging in via SSH from the host it self:

[[email protected] ~]# ssh 192.168.122.85 -l root
The authenticity of host '192.168.122.85 (192.168.122.85)' can't be established.
RSA key fingerprint is 65:62:aa:98:89:c1:57:19:1b:f9:aa:97:cf:7c:14:49.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.122.85' (RSA) to the list of known hosts.
[email protected]'s password: 
Last login: Sat May 24 19:38:13 2014
[[email protected] ~]# ip -4 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
	inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
	inet 192.168.122.85/24 brd 192.168.122.255 scope global eth0

Lastly to make sure the NAT is working properly, check to see that you can reach the outside network from the VM itself:

[[email protected] ~]# ping google.com -c 3
PING google.com (74.125.239.100) 56(84) bytes of data.
64 bytes from nuq05s01-in-f4.1e100.net (74.125.239.100): icmp_seq=1 ttl=53 time=41.1 ms
64 bytes from nuq05s01-in-f4.1e100.net (74.125.239.100): icmp_seq=2 ttl=53 time=39.1 ms
64 bytes from nuq05s01-in-f4.1e100.net (74.125.239.100): icmp_seq=3 ttl=53 time=41.2 ms

--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2045ms
rtt min/avg/max/mdev = 39.183/40.519/41.234/0.945 ms

Everything looks good.

<div class='shareaholic-canvas' data-app='share_buttons' data-app-id='6778737' style="float: left"></div>

<div class="pagination" style="float: right">
  <ul>
  
    <li class="prev"><a href="/2014/05/update-zabbix-2-2-monitor-vmware/" title="Update Zabbix to 2.2 and Monitor VMware">&larr; Previous</a></li>
  
    <li><a href="/archive.html">Archive</a></li>
  
    <li class="next"><a href="/2014/05/migrate-from-wordpress-to-jekyll" title="Migrate From Wordpress to Jekyll With Github Pages">Next &rarr;</a></li>
  
  </ul>
</div>


<hr>
<script async src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script> <!-- virtuallyhyper.com --> <ins class="adsbygoogle"
 style="display:inline-block;width:728px;height:90px"
 data-ad-client="ca-pub-7439655074445804"
 data-ad-slot="4448070391"></ins> <script> (adsbygoogle = window.adsbygoogle || []).push({}); </script>

blog comments powered by Disqus

` is used to inject content into the web page.

_posts Your dynamic content, so to speak. The naming convention of these files is important, and must follow the format: YEAR-MONTH-DAY-title.MARKUP. The permalinks can be customized for each post, but the date and markup language are determined solely by the file name.
_data Well-formatted site data should be placed here. The jekyll engine will autoload all yaml files (ends with .yml or .yaml) in this directory. If there’s a file members.yml under the directory, then you can access contents of the file through site.data.members.
_site This is where the generated site will be placed (by default) once Jekyll is done transforming it. It’s probably a good idea to add this to your .gitignore file.
index.html and other HTML, Markdown, Textile files Provided that the file has a YAML Front Matter section, it will be transformed by Jekyll. The same will happen for any .html, .markdown, .md, or .textile file in your site’s root directory or directories not listed above.
Other Files/Folders Every other directory and file except for those listed above—such as css and images folders, favicon.ico files, and so forth—will be copied verbatim to the generated site. There are plenty of sites already using Jekyll if you’re curious to see how they’re laid out.

Jekyll Bootstrap

Most people just take a copy of an existing blog and go from there, since you don’t really want to create the above layout by hand. A lot of jekyll powered sites are listed in Jekyll Sites. If you like one just clone it. To get started with a Jekyll site, there is a a project called JekyllBootstrap. It basically eases the process of managing a Jekyll site. To create the first jekyll site, check out the instructions laid out in Jekyll QuickStart. So let’s try it out, first let’s clone their github repo:

[email protected]:~$git clone https://github.com/plusjade/jekyll-bootstrap.git
Cloning into 'jekyll-bootstrap'...
remote: Reusing existing pack: 2062, done.
remote: Total 2062 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (2062/2062), 811.25 KiB | 0 bytes/s, done.
Resolving deltas: 100% (790/790), done.
Checking connectivity... done.

Now let’s build and serve that jekyll-bootstrap template locally:

[email protected]:~$cd jekyll-bootstrap/
[email protected]:~/jekyll-bootstrap$jekyll serve -w
Configuration file: /Users/elatov/jekyll-bootstrap/_config.yml
       Deprecation: The 'pygments' configuration option has been renamed to 'highlighter'. Please update your config file accordingly. The allowed values are 'rouge', 'pygments' or null.
            Source: /Users/elatov/jekyll-bootstrap
       Destination: /Users/elatov/jekyll-bootstrap/_site
      Generating...
                    done.
 Auto-regeneration: enabled
    Server address: http://0.0.0.0:4000/
  Server running... press ctrl-c to stop.

Now if you point your browser to http://localhost:4000 you should see the template:

jekyll-running-example-bootstrap

Jekyll Themes

There are a bunch of themes out there. Here are a couple of pages that have themes:

Like I mentioned before, you can either clone the whole project or you can actually use JekyllBootstrap to install a theme. For example here is an easy way to install the twitter-bootstrap based theme with JekyllBootstrap:

[email protected]:~/jekyll-bootstrap$rake theme:install git="https://github.com/jekybootstrap/theme-twitter.git"
Cloning into './_theme_packages/_tmp'...
remote: Reusing existing pack: 26, done.
remote: Total 26 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (26/26), done.
Checking connectivity... done.
mv ./_theme_packages/_tmp ./_theme_packages/twitter
./_includes/themes/twitter/default.html already exists. Do you want to overwrite? [y/n] y
mkdir -p ./_includes/themes/twitter
cp -r ./_theme_packages/twitter/_includes/themes/twitter/default.html ./_includes/themes/twitter/default.html
./_includes/themes/twitter/page.html already exists. Do you want to overwrite? [y/n] y
mkdir -p ./_includes/themes/twitter
cp -r ./_theme_packages/twitter/_includes/themes/twitter/page.html ./_includes/themes/twitter/page.html
./_includes/themes/twitter/post.html already exists. Do you want to overwrite? [y/n] y
mkdir -p ./_includes/themes/twitter
cp -r ./_theme_packages/twitter/_includes/themes/twitter/post.html ./_includes/themes/twitter/post.html
./_includes/themes/twitter/settings.yml already exists. Do you want to overwrite? [y/n] y
mkdir -p ./_includes/themes/twitter
cp -r ./_theme_packages/twitter/_includes/themes/twitter/settings.yml ./_includes/themes/twitter/settings.yml
mkdir -p ./assets/themes/twitter/css/1.4.0
cp -r ./_theme_packages/twitter/assets/themes/twitter/css/1.4.0/bootstrap.css ./assets/themes/twitter/css/1.4.0/bootstrap.css
./assets/themes/twitter/css/style.css already exists. Do you want to overwrite? [y/n] y
mkdir -p ./assets/themes/twitter/css
cp -r ./_theme_packages/twitter/assets/themes/twitter/css/style.css ./assets/themes/twitter/css/style.css
=> twitter theme has been installed!
=> ---
=> Want to switch themes now? [y/n] y
Generating 'twitter' layout: default.html
Generating 'twitter' layout: page.html
Generating 'twitter' layout: post.html
=> Theme successfully switched!
=> Reload your web-page to check it out =)

Now if you re-run jekyll you should see the following after you visit http://localhost:4000 :

jekyll-with-twitter-theme-running

Customize Jekyll Configurations

Let’s add the information regarding the site. I ended up modifying the following lines in the _config.yml file:

[email protected]:~/jekyll-bootstrap$git diff
diff --git a/_config.yml b/_config.yml
index 17bd3e2..7f185e3 100644
--- a/_config.yml
+++ b/_config.yml
@@ -1,21 +1,19 @@
 # This is the default format.
 # For more see: http://jekyllrb.com/docs/permalinks/
-permalink: /:categories/:year/:month/:day/:title
+permalink: /:year/:month/:title

 exclude: [".rvmrc", ".rbenv-version", "README.md", "Rakefile", "changelog.md"]
-pygments: true
+highligher: pygments

 # Themes are encouraged to use these universal variables
 # so be sure to set them if your theme uses them.
 #
-title : Jekyll Bootstrap
-tagline: Site Tagline
+title : My Blog
+tagline: Just a Blog
 author :
-  name : Name Lastname
-  email : [email protected]
-  github : username
-  twitter : username
-  feedburner : feedname
+  name : Me Moxz
+  email : [email protected]
+  github : moxz1

 # The production_url is only used when full-domain names are needed
 # such as sitemap.txt
@@ -25,7 +23,7 @@ author :
 # Else if you are pushing to username.github.io, replace with your username.
 # Finally if you are pushing to a GitHub project page, include the project name at the end.
 #
-production_url : http://username.github.io
+production_url : http://moxz1.github.io

On the main page, I decided to just list posts and nothing else. Here is what I ended up with in the index.md file:

[email protected]:~/jekyll-bootstrap$cat index.md
---
layout: page
title: Posts
---
{% include JB/setup %}

<ul class="posts">
{% for post in site.posts  limit:10 %}
    <a href="{{ BASE_PATH }}{{ post.url }}"><h3> {{ post.title }}<br /></h3></a>
	<i>{{ post.date | date_to_string }}<br /></i>
        {{ post.content | strip_html | truncatewords:75}}
            <a href="{{ post.url }}">Read more...</a>
    {% endfor %}
</ul>

After a reload of jekyll, I saw the following on the local site:

jekyll-personal-info-added

If you like how it looks, you can push it to the github pages:

[email protected]:~/jekyll-bootstrap$git remote set-url origin https://[email protected]/moxz1/moxz1.github.io.git
[email protected]:~/jekyll-bootstrap$git add --all
[email protected]:~/jekyll-bootstrap$git commit -m "updates"
[master f692c6f] updates
10 files changed, 482 insertions(+), 197 deletions(-)
create mode 100644 assets/themes/twitter/css/1.4.0/bootstrap.css
rewrite assets/themes/twitter/css/style.css (91%)
rewrite index.md (94%)
[email protected]:~/jekyll-bootstrap$git push origin master --force
Counting objects: 38, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (19/19), done.
Writing objects: 100% (21/21), 11.15 KiB | 0 bytes/s, done.
Total 21 (delta 5), reused 8 (delta 0)
To https://[email protected]/moxz1/moxz1.github.io.git
 1f3596c..f692c6f  master -> master

Then after some time, if you visit the github user pages, you will see the same site:

github-updates-pushed-and-live

Migrate Wordpress Posts to Jekyll

There are a couple of methods to the approach. Here are a few:

I ended up using the bottom one. After you install the plugin in your wordpress install, you can either go to the Wordpress Managament Page and you will see the Export to Jekyll button there:

export-to-jekyl-plugin-wp

Upon clicking that, it will start the export and you should get a zip of the export. Mine kept timing out, so I did it manually on the host it self:

$ cd /var/www/wp-content/plugins/wordpress-to-jekyll-exporter
$ php jekyll-export-cli.php > ~/jekyll-export.zip

I copied the zip from the host and here were the contents of the zip extracted:

[email protected]:~/jek$ tree -L 1 jekyll-export
jekyll-export
├── about
├── _config.yml
├── contact
├── _posts
└── wp-content

4 directories, 1 file

wp-contents contained all the uploads from wordpress:

[email protected]:~/jek$ls -l jekyll-export/wp-content/
total 0
drwxr-xr-x  10 elatov  staff  340 May 30 09:57 uploads

and _posts, is a directory with all the converted posts:

[email protected]:~/jek$ls -l jekyll-export/_posts
total 376
-rw-r--r--  1 elatov  staff  71062 May 30 09:59 2014-04-22-post1.md
-rw-r--r--  1 elatov  staff  26118 May 30 09:59 2014-05-05-post2.md
-rw-r--r--  1 elatov  staff  87552 May 30 09:59 2014-05-06-post3.md

You can just copy those over to your jekyll setup:

[email protected]:~$rsync -avzP jek/jekyll-export/_posts/. moxz1.github.io/_posts/.
building file list ...
4 files to consider
./
2014-04-22-post1.md
       71062 100%   18.26MB/s    0:00:00 (xfer#1, to-check=2/4)
2014-05-05-post2.md
       26118 100%    8.30MB/s    0:00:00 (xfer#2, to-check=1/4)
2014-05-06-post3.md
       87552 100%    8.35MB/s    0:00:00 (xfer#3, to-check=0/4)

sent 49675 bytes  received 92 bytes  99534.00 bytes/sec
total size is 184732  speedup is 3.71

Now launching your local jekyll instance:

[email protected]:~/moxz1.github.io$jekyll serve -w
Configuration file: /Users/elatov/moxz1.github.io/_config.yml
            Source: /Users/elatov/moxz1.github.io
       Destination: /Users/elatov/moxz1.github.io/_site
      Generating...
                    done.
 Auto-regeneration: enabled
Configuration file: /Users/elatov/moxz1.github.io/_config.yml
    Server address: http://0.0.0.0:4000/
  Server running... press ctrl-c to stop.

You can visit the local site (http://localhost:4000), and you will see your posts included in the main page:

posts_migrated_tojekyll-local-view

Clean up Converted Markdown

None of the above converters are perfect, and after the migration you will definitely end up with some left over HTML. Here are a couple of sites that help with clean up:

The first one has an R Script which clean up a bunch of HTML tags and the second one has a ruby script to clean up UTF-8 encoded characters. I ended playing with sed (gsed from macports) for some conversions. Here are a couple of that I ran:

### clean up code snippets
for file in $(grep -E '\[code|\[shell|\[bash|\[powershell|\[xml' * | cut -d : -f 1 | uniq); do echo $file; gsed -ri '/\[code|\[shell|\[bash|\[powershell|\[xml/,/\[\/code|\[\/shell|\[\/bash|\[\/powershell|\[\/xml\]/{s/^/\t/g}' $file; done
for file in $(grep -E '\[code|\[shell|\[bash|\[powershell|\[xml' * | cut -d : -f 1 | uniq); do echo $file; gsed -ri 's/\[[code|bash|powershell|xml|shell].*\]//g' $file; done
for file in $(grep -E '\[\/code\]|\[\/shell\]|\[\/bash\]|\[\/powershell\]|\[\/xml\]' *| cut -d : -f 1 | uniq); do echo $file; gsed -ri 's/\[\/[code|bash|powershell|xml|shell]*\]//g' $file; done

### Replace single quote
for i in $(grep '&#8217;' * | awk -F : '{print $1}'  | uniq); do echo $i;gsed -i "s/&#8217;/'/g" $i;  done

### replace _\ with _
for i in $(grep '\_' * | awk -F : '{print $1}'  | uniq); do echo $i; done

### replace the &gt; with >
for i in $(grep '&gt;' * | awk -F : '{print $1}'  | uniq); do echo $i; gsed -i 's%\&gt;%>%g' $i; done

### Replace Smiley images with characters
for i in $(grep '<img src="http://site.com/wp-includes/images/smilies/icon_smile.gif"' * | awk -F : '{print $1}'  | uniq); do echo $i; gsed -i 's%<img src="http://virtuallyhyper.com/wp-includes/images/smilies/icon_smile.gif".*/>%:\)%g' $i; done

There were a bunch more but you get the point. The reason why I did it by hand was to make sure everything was converted as I expected. This was tedious, but it was better than re-writing all the posts and at the end I knew the markdown files were clean.

Most of the href links looked like this in the new files:

<a href="https://communities.vmware.com/thread/423099" onclick="javascript:_gaq.push(['_trackEvent','outbound-article','http://communities.vmware.com/thread/423099']);">ESXi 5.x on new Apple Mac Mini 6,2 Late 2012</a>

To fix those, I wrote a little python script:

[email protected]:~$cat conv-html-url-to-md.py
import fileinput
import re

for line in fileinput.input(inplace=1):
    line = re.sub(r'<a href="(.*)" onclick="(.*)">(.*)</a>', r'[\3](\1)', line.rstrip())
    print(line)

Then I just ran the following to clean up all the links:

[email protected]:~/moxz1.github.io$ for i in $(ls | grep -v py); do echo $i; python conv-html-url-to-md.py $i; done

The image links were in the following format in the converted files:

<a href="http://site.com/wp-content/uploads/2014/04/installing-unetbootin.png" onclick="javascript:_gaq.push(['_trackEvent','outbound-article','http://virtuallyhyper.com/wp-content/uploads/2014/04/installing-unetbootin.png']);"><img src="http://virtuallyhyper.com/wp-content/uploads/2014/04/installing-unetbootin.png" alt="installing unetbootin ESXi on MacMini 6,2" width="520" height="369" class="alignnone size-full wp-image-10454" title="ESXi on MacMini 6,2" /></a>

Now with another python script we can fix the links for the images:

[email protected]:~$cat conv-html-url-to-md.py
import fileinput
import re

for line in fileinput.input(inplace=1):
    line = re.sub(r'<a href="(.*)" onclick=(.*) title="(.*)" \/></a>', r'![\3](\1)', line.rstrip())
    print(line)

Then with another for loop, we can run that through all the posts:

[email protected]:~/moxz1.github.io$ for i in $(ls | grep -v py); do echo $i; python conv-html-url-to-md.py $i; done

Moving Images and Uploads

The first thing you will notice in the uploads directory is that there are multiple versions of the same image:

[email protected]:~$ls jek/jekyll-export/wp-content/uploads/2012/03/ | tail -9
Ubuntu_11_10_top-150x150.png
Ubuntu_11_10_top-300x55.png
Ubuntu_11_10_top-620x163.png
Ubuntu_11_10_top.png
esxtop_latency-150x150.png
esxtop_latency-300x222.png
esxtop_latency-620x200.png
esxtop_latency-927x180.png
esxtop_latency.png

To clean those up we can use find, here the command to list them:

[email protected]:~/jek/jekyll-export/wp-content/uploads/2012/03$find . -name "*-[0-9]*x[0-9]*.png"
./esxtop_latency-150x150.png
./esxtop_latency-300x222.png
./esxtop_latency-620x200.png
./esxtop_latency-927x180.png
./Ubuntu_11_10_top-150x150.png
./Ubuntu_11_10_top-300x55.png
./Ubuntu_11_10_top-620x163.png

Then to clean them up, just run the following:

[email protected]:~/jek/jekyll-export/wp-content/uploads/2012/03$find . -name "*-[0-9]*x[0-9]*.png" -delete
[email protected]:~/jek/jekyll-export/wp-content/uploads/2012/03$find . -name "*-[0-9]*x[0-9]*.png"

For an easy example I will use github to host my images, but I would recommend using some cloud storage alternative. Here is a pretty good list of options. After you create a new repository(I just called mine uploads and I put the whole uploads directory from the jekyll export), you can access your files with the raw links. This was discussed at:

So for example the above image which was located at:

wp-content/uploads/2012/03/esxtop_latency.png

From the Jekyll Export, will be accessible in the github repository with the following URL:

https://github.com/moxz1/uploads/raw/master/2012/03/esxtop_latency.png

Now we can use the following to point all the image links to the github repository:

[email protected]:~$cat conv-html-url-to-md.py
import fileinput
import re

for line in fileinput.input(inplace=1):
    line = re.sub(r'http\:\/\/site.com\/wp-content\/uploads/(.*)', r'https://github.com/moxz1/uploads/raw/master/\1', line.rstrip())

After the image links were updated, I actually ended up running a python-based linkchecker on the jekyll site just to make sure nothing came up broken. To do that I first install pip for python:

[email protected]:~$sudo port install py27-pip

Then I made the 2.7 version be the default one:

[email protected]:~$sudo port select --set pip pip27

Lastly I searched for the python package:

[email protected]:~$pip search pylink
pylinkgrammar             - Python bindings for Link Grammar system
PyLink                    - Universal communication interface using File-Like API
pylinkmobile              - Link Mobile Solutions API wrapper
pylinkchecker             - Simple crawler that detects link errors such as 404 and 500.
PyLinkedIn                - Client for LinkedIn API
pylinktester              - a link tester written in python

Then to install it:

[email protected]:~$sudo pip install pylinkchecker

Then I ran the following to check for broken links:

[email protected]:~$pylinkcheck.py http://localhost:4000 -O -o pylink.txt

It look about 15 minutes to finish, but then I was able to see if any of my links to images on github were broken:

[email protected]:~$grep github pylink.txt
  not found (404): https://github.com/moxz1/uploads/raw/master/2013/02/zenoss-ssh-linux-device.png
  not found (404): https://github.com/moxz1/uploads/raw/master/2013/04/vm_details_change_video_to_glx.png

I had a very small amount and I fixed them really quick. Just as a side note another good linkchecker can be found here, it’s also based on python. That one can be setup to run from cron to check for any broken links on your site.

Writing new Posts

With jekyllbootstrap, we can use the prebuilt Rakefile to create new posts. Most of the instructions are laid out in Jekyll QuickStart. For example here is a quick shortcut to create a new post:

[email protected]:~/moxz1.github.io$rake post title="New Post"
Creating new post: ./_posts/2014-05-30-new-post.md

You will see a template generated for the new post:

[email protected]:~/moxz1.github.io$cat _posts/2014-05-30-new-post.md
---
layout: post
title: "New Post"
description: ""
category:
tags: []
---
{% include JB/setup %}

The top section is the metadata of the post and you can define tags and categories of the post if you want. Here is a list of available variables from Jekyll Front-matter:

VARIABLE DESCRIPTION
layout If set, this specifies the layout file to use. Use the layout file name without the file extension. Layout files must be placed in the _layouts directory.
permalink If you need your processed blog post URLs to be something other than the default /year/month/day/title.html then you can set this variable and it will be used as the final URL.
published Set to false if you don’t want a specific post to show up when the site is generated.
category categories Instead of placing posts inside of folders, you can specify one or more categories that the post belongs to. When the site is generated the post will act as though it had been set with these categories normally. Categories (plural key) can be specified as a YAML list or a space-separated string.
tags Similar to categories, one or multiple tags can be added to a post. Also like categories, tags can be specified as a YAML list or a space- separated string.

After you define the metadata of the post, you can use your favorite markdown editor to write the content:

Here is a screenshot of Mou:

Mou-Example

Here is Haroopad:

haroopad-example

I like Mou and Haroopad for their shortcuts, here are some shortcuts from haroopad:

haroopad-keyboard-shortcuts.png

You can see the full list under the help section of haroopad or Mou. Here is Sublime Text 3:

sublime-text-preview.png

Sublime Text Plugins don’t have the variety of shortcuts by default, but you can definitely customize it to your need. While you are editing the file, upon saving the file to check your changes, you will see jekyll letting you know if a file has changed:

[email protected]:~/moxz1.github.io$jekyll serve -w
Configuration file: /Users/elatov/moxz1.github.io/_config.yml
            Source: /Users/elatov/moxz1.github.io
       Destination: /Users/elatov/moxz1.github.io/_site
      Generating...
                    done.
 Auto-regeneration: enabled
Configuration file: /Users/elatov/moxz1.github.io/_config.yml
    Server address: http://0.0.0.0:4000/
  Server running... press ctrl-c to stop.
      Regenerating: 1 files at 2014-05-30 14:43:43 ...done.
      Regenerating: 2 files at 2014-05-30 14:43:48 ...done.

After you are done editing your post and you confirmed the local copy looks good, I would run the following to commit the changes to your github pages:

[email protected]:~/moxz1.github.io$jekyll build --safe
Configuration file: /Users/elatov/moxz1.github.io/_config.yml
            Source: /Users/elatov/moxz1.github.io
       Destination: /Users/elatov/moxz1.github.io/_site
      Generating...
                    done.
[email protected]:~/moxz1.github.io$git add --all
[email protected]:~/moxz1.github.io$git commit -m 'new post'
[master 24ea8b1] new post
 1 file changed, 1 insertion(+), 2 deletions(-)
[email protected]:~/moxz1.github.io$git push origin master
Counting objects: 7, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 368 bytes | 0 bytes/s, done.
Total 4 (delta 3), reused 0 (delta 0)
To https://[email protected]/moxz1/moxz1.github.io.git
   d254dae..24ea8b1  master -> master

After that you should see your new post on your github pages:

github-user-pages-with-new-post

I did a test build just in case with the –safe flag, because that is how github runs jekyll. From Jekyll Plugins:

Plugins on GitHub Pages

GitHub Pages is powered by Jekyll, however all Pages sites are generated using the --safe option to disable custom plugins for security reasons. Unfortunately, this means your plugins won’t work if you’re deploying to GitHub Pages.

and from Troubleshooting GitHub Pages build failures:

To view Jekyll build errors locally, install Jekyll on your computer and run the jekyll build --safe command in the root of your GitHub Pages repository.

This way we can check for any errors before pushing anything to the github pages.

Create New Post with prose.io

There is also an online tool that allows you to create posts prose.io. After visiting the above page and authorizing prose.io to access your github pages, you will see the following:

prose-io-first-page

Click on the project will show you the contents:

prose-io-inside-project

After clicking going inside the _posts directory and clicking New File, you can give the post a title and enter markdown code:

prose-io-new-file.png

You can also get a preview of the page after the markdown is parsed:

prose-io-preview-post

You can also edit the metadata of the post from here (by default, it’s blank):

prose-io-edit-metadata

You can also click on Submit Changes to push the change to github:

prose-io-save-changes

To publish the post, click on the “Unpublish” button:

prose-io-publish-post

and then click Submit Changes one more time. After that if you go back to your github pages you will see the new post:

post-with-prose-published

Jekyll Summary

I liked the transition to the new setup. Being a persion who loves the command-line, it’s perfect. I think the only downside to this, is the fact that you have to check out the whole site from github before making any changes (and that means that you have to have a local copy on multiple machines). Also running Jekyll locally can be process consuming, depending on how many posts it has go through to generate the site. It will definitely give you more control over your content, and it’s up to you to stay organized. There are definitely a lot pros to the setup as well. You don’t have to manage your own jekyll server, github hosts the pages for you. Also since github is a version control system, you basically have backups of each commit that you make to github. This way of managing a site is definitely not for everyone. Here are some examples of people that had a different experience with jekyll.


blog comments powered by Disqus