Setting up Varnish with Virtualmin – Centos in 5 steps

If you find this free website useful – why don’t you support this with a donation? It is easy…. read more ….

I have just finished setting up Varnish on my test server that uses Webmin/Virtualmin, with Apache on Centos.

It has been a little bit of a voyage of discovery but I think I have cracked it. Much of what I have here is based on many other blogs and forums. In essence once you know how its easy, but there are a few pit holes to watch for in Virtualmin/Webmin, as editing the underlying httpd.conf file isn’t the way to go.

The main steps are

  1. Get Varnish
  2. Set up the Varnish control file
  3. Modify the Apache virtual hosts port number
  4. Restart the servers
  5. Tidy up Virtualmin server templates and make sure that Varnish runs on boot / restarted with Apache

and you are done.  My experience is on a Centos server, I’m sure that the process is the same on Ubuntu except the details of some of the linux level commands and file locations

1. Get Varnish

The easy way – from a repository

yum install varnish

2. Set up the Varnish control file & process defaults

In essence to get up and running this is simply putting the port and IP address that Apache will be listening on.     Normally Apache listens on the http port 80, but as Varnish sits in front of Apache, Apache and Varnish will listen on 80 apache has to move – the ‘default’ documentation moves Apache to 8080.

Some people write that they can set it up with Apache listening on 127.0.0.1:80 and Varnish on myexternalIP:80  but I couldn’t get this to work and wasted much time trying to work out where the conflict comes from. If you don’t want to waste time stick to 80 and 8080.

The default.vcl is in etc/varnish

There are a lot more advanced VCL things that you might have to do, like excluding some virtual hosts, and passing through the originators IP address (that Varnish looses as it is a proxy) I’ll cover them in my ‘advanced’ VCL musing in a later post.

You may will need to go into /etc/sysconfig/varnish  and set the listen port

3. Modify the Apache configuration to use 8080

All the advice points to updating /etc/httpd/conf/httpd.conf    but if you are running Virtualmin/Webmin stop – don’t do that.

You have to modify things via Virtualmin or you will get problems later.

It took a while to find

Virtualmin>[server]>Server Configuration>Change IP address and change the ‘New HTTP Port’ from 80 to 8080. Do this for every existing virtual server

If you have a lot, then you can do this at command level, I wrote this script that goes through all virtual server and changes the port to 8080

for dom in $(virtualmin list-domains –with-feature web –name-only); do
virtualmin modify-web –domain $dom –port 8080
done

 

You then just need to make sure that apache isn’t also listening on port 80.

Go to Webmin>Servers>Apache Webserver, scan down the list of existing Virtual Servers they should all be 8080, the switch to the Global configuration tab > Networking and Addresses, under ports if the is an entry with 80 remove it (none and blank) and save, port 80 shouldn’t be mentioned here at all.

4. Restart The Servers

Sounds simple. I tend to prefer stopping and starting servers at command level. You just need to make sure you stop apache, start varnish and then start apache

e.g.

service httpd stop

service varnish start

service httpd start

 

If httpd doesn’t start then its because it is still listening to post 80, or you didn’t change the varnish default.vcl from 80 to 8080

Check you sites are running!

You also might want to see if varnis is actually running and doing something – a couple of command line commands to run woudl be varnishlog and varnishstat. I might post a bit more about mnitoring when I get more familiar with it.

5. Tidy up Virtualmin server templates and make sure that Varnish runs on boot / restarted with Apache

When you create new virtual servers it uses a server template, and if you don’t change the values you will create new virtual servers on port 80 which will create issues.

Virtualmin>Server Templates> edit each template / template section = Apache website, change Port number for Virtual Hosts to 8080

You need to make sure that Varnish start on reboot too, so

Webmin>System>Bootup and Shutdown

scroll down to find Varnish, edit and set it to start on boot

And you will also need to ensure that Varnish is restarted after you restart Apache for any reason (well you may get away with not, but I think it is safer)

Webmin>Servers>Apache Webserver

follow the top link to module config and the pull down, system configuration  and modify the command to start Apache from  /etc/rc.d/init.d/httpd start    to /etc/rc.d/init.d/httpd start;/etc/init.d/varnish restart

and the same for the command to apply after configuration.

That should do it!

references / sources

In my voyage of discovery I referenced these items the most

http://blog.nexcess.net/2012/05/21/installing-varnish-for-your-magento-store-on-centos-6/  despite being about Magneto, this well written resource cover a lot of Centos setup in a simple and informative way. Watch out, don’t just cut and paste as this blog has converted quotes into html format.

https://www.varnish-cache.org/docs/3.0/ is the official Varnish documentation which is fine, but it doesn’t help too much beyond the basic setup, worth reading through

http://ocaoimh.ie/2011/08/09/speed-up-wordpress-with-apache-and-varnish/ this is a Ubuntu based post, focusing on WordPress and uses the local host sharing 80 port concept, this actually wasted a lot of time for me. I am also a bit suspect of the wordpress based vcl, but I did base my vcl on some of it.

https://www.varnish-cache.org/trac/wiki/VCLExamples  has VCL examples, and is a bteer place to get to understand the VCL options and there are some user contributed WordPress VCls which may be  a better way of going than the above.


Posted

in

, ,

by

Comments

13 responses to “Setting up Varnish with Virtualmin – Centos in 5 steps”

  1. Martin avatar

    Gee I wish I’d found this article a few days ago. Would have saved me the same grief since I’m not a SysAdmin 🙂
    Now to figur out why switching to fcgid in VirtualMin consumes twice the memory. Maybe I’ve got some double up running or something.
    Nice summary, mate

    1. Badly Wired avatar
      Badly Wired

      I have been thinking about the fcgid issue a lot too. It seems that you get a process forked for each php user. If you get some further thoughts please share them.

      I Impelemented OPCache on my servers (the replacement for APC), so at least the common PHP code is cached.

  2. Guy avatar
    Guy

    Hi there,

    Just wanted to say thank you for taking the time to write this up as I have been struggling.

    About to launch in and try again.

    Cheers

    Guy

  3. Serhat avatar
    Serhat

    This is a great help, thanks for it. Just a side note for anyone using debian like me: the default configuration for varnish on debian is on /etc/default/varnish and not on /etc/sysconfig/varnish as on CentOs.

    1. Badly Wired avatar
      Badly Wired

      Thanks for you additional input. Yes, I operate my servers on Centos, so my articles are Centos centric. Hopefully your comments can help people on Debian.

  4. Eddie avatar
    Eddie

    Very useful, thank you!

    Why did you remove the virtualmin script to batch change the ports apache listens on?

    1. Badly Wired avatar
      Badly Wired

      Hi Eddie. I don’t quite understand the question about the batch script, it is still there as far as I can see. Are you gee ting an issue viewing it? Please let me know.
      Alan

  5. Eddie avatar
    Eddie

    sorry, i got confused. the article which contains the script is /www.tienle.com/2013/10-08/setting-varnish-with-virtualmin-centos-5-steps.html

    i have multiple vhosts in different IPs in the same physical server. i’m trying to figure out the “backend default” syntax in default.vcl

    1. Badly Wired avatar
      Badly Wired

      Thanks, cheeky that Tienle’s copied my blog word for word!!! they have added a little bit more abut benchmarking. As he stole my stuff do you think I should nick his?

  6. SR avatar
    SR

    Was the article removed? I don’t see the 5 steps.

    1. Badly Wired avatar
      Badly Wired

      No the article is still there. If you look below the first paragraph you will see a box saying
      “to read all of this post we would really appreciate you liking socially with one of these buttons”

      If you facebook like or tweet or whatever the full article gets reveal. It is known as ‘pay with a like’, lot cheaper than paying with money 🙂

  7. John Byrd avatar

    Your HTML reformatter reformatted your virtualmin shell code example. Try instead:

    for dom in ($virtualmin list-domains –with-feature web –name-only); do
    virtualmin modify-web –domain $dom –port 8080
    done

    1. Alan avatar
      Alan

      Thanks. The ‘backticks’ do actually work in centos but your syntax is probably better as it should work on any bash shell. Although you made a typo in you ‘correction’. It should have the $ before the bracket (I’ll modify the code in the post)

      for dom in $(virtualmin list-domains –with-feature web –name-only); do
      virtualmin modify-web –domain $dom –port 8080
      done

Leave a Reply to Eddie Cancel reply

Your email address will not be published. Required fields are marked *