nginxでDDoS対策をする方法

nginxでDDoS対策をするための方法を調べてみました。
いくつか方法が見つかったので、リンクをまとめてみました。あとで、色々試してみようと思います。
nginxをプロキシサーバとして利用して、バックエンドにアプリサーバを配置するような構成で使うので、外部から来たDDoSをnginxでブロックしてアプリサーバを守る。といったような用途になるかと思います。
もちろん、nginxが応答できるHTTPリクエストの範疇に収まらない攻撃は、その前段階でブロックしたり、他のサーバにアクセスを逃したりする必要があります。
DDoS攻撃に対するシンプルな戦略 | ツチノコブログ(http://tsuchinoko.dmmlabs.com/?p=611)
さらに、こちらで紹介されているように、完璧に防ごうとすると青天井のコストが見えてしまったりするので、”どれくらいDDoSからの保護をするのか”を明確にルールを決めておくと良さそうですね。

以下、リンク集です。

nginx + mruby + http-dos-detector
http-dos-detector
https://github.com/matsumoto-r/http-dos-detector
nginxでもapacheでも使用可能なDoS的アクセスを検知して任意の制御をするWebサーバ拡張をmrubyで作った - 人間とウェブの未来
http://hb.matsumoto-r.jp/entry/2015/06/08/224827
ngx_mrubyでのdos_detector(http-dos-detector)をためし、パフォーマンスに与える影響を測った - Qiita　
http://qiita.com/sawanoboly/items/74368e002631bed3afb7
Mitigating DDoS Attacks nginx + ngx_mruby + http-dos-detector
http://takeshiyako.blogspot.jp/2015/11/mitigating-ddos-attacks-nginx-ngxmruby.html

nginx + mruby + mruby-cgroup
mruby-cgroup
https://github.com/matsumoto-r/mruby-cgroup
RubyKaigi 2014 mod_mruby ngx_mruby // Speaker Deck
https://speakerdeck.com/matsumoto_r/rubykaigi-2014-mod-mruby-ngx-mruby
人間とウェブの未来 - mruby-cgroupとmod_mrubyでApacheのリソースを制御、そこから得られるcgroupの挙動とは
http://blog.matsumoto-r.jp/?p=3285
mod_mruby で特定のCGIのCPU使用率を制限してみたら感動した話 - フリーエンジニアライフ
http://takeyuweb.hatenablog.com/entry/2014/11/22/120335

nginx + nginx-maxconn-module
nginx-maxconn-module
https://github.com/cybozu/nginx-maxconn-module
nginx の拡張モジュールを書いて DoS 対策をした - Cybozu Inside Out | サイボウズエンジニアのブログ
http://blog.cybozu.io/entry/8363

nginx conf
nginx - ELB内のngixでDDoS対策 - Qiita
http://qiita.com/drobune/items/fce3e3629a472dc3237e
Nginx Tuning For Best Performance
https://gist.github.com/denji/8359866#nginx-simple-ddos-defense
Nginxで使える制限一覧 | しのぶら跡地
http://shinobra.com/2012/03/sawanoboly/nginx_limiter
コーポレートサイトのNginxの設定をチューニングも兼ねていじった - onigra.github.io
http://onigra.github.io/blog/2015/01/07/corporate-site-tuning-with-nginx/
Mitigating DDoS Attacks nginx with module
http://takeshiyako.blogspot.jp/2015/11/mitigating-ddos-attacks-nginx-module.html


AWSを使っているならば下記のスライドが参考になります。
nginxのログからブラックリストを作ってアクセスコントロールに追加したりとかは、お手軽にできそうですね。
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
http://www.slideshare.net/AmazonWebServices/ddos-resiliency-with-amazon-web-services-sec305-aws-reinvent-2013
(SEC307) Building a DDoS-Resilient Architecture with Amazon Web Services | AWS re:Invent 2014
http://www.slideshare.net/AmazonWebServices/sec307-building-a-ddosresilient-architecture-with-amazon-web-services-aws-reinvent-2014
AWSでブラックリスト型のFirewallを実装する。〜BlackList Firewallパターン〜 - プログラマになりたい
http://blog.takuros.net/entry/20131213/1386944370



Top image from BlinkenArea.org 28c3-ddos-use.more.bandwith

PHP with Amazon DynamoDB

This blog entry aim to construct the good connectivity performance from PHP to Amazon DynamoDB.

Make DynamoDB table
Table Name: ProductCatalog
Primary Hash Key: Id (String)
Provisioned Read Capacity Units: 10000
Provisioned Write Capacity Units: 10000
Region: Asia Pacific (Tokyo)

Launch EC2 Instance
AMI ID: Amazon Linux AMI 2015.09 (HVM), SSD Volume Type - ami-9a2fb89a
Instance type: c4.xlarge

Install PHP and AWS SDK for PHP
Install PHP5.6

# yum -y install php56 php56-devel php56-fpm php56-opcache gcc

Make app dir and install new version of aws-sdk-php by Composer.

# mkdir ~/sample_app; cd ~/sample_app
# curl -sS https://getcomposer.org/installer | php
# php composer.phar require aws/aws-sdk-php
# php composer.phar require doctrine/cache
# php composer.phar install --optimize-autoloader
# yum -y install php-pear
# pecl install uri_template
# echo "extension=uri_template.so" >> /etc/php.ini
# php composer.phar show --installed | grep aws/aws-sdk-php
aws/aws-sdk-php        3.9.1 AWS SDK for PHP - Use Amazon Web Services in your PHP project

Make AWS credentials file.

# mkdir ~/.aws/
# emacs ~/.aws/credentials
[default]
aws_access_key_id = YOUR_AWS_ACCESS_KEY_ID
aws_secret_access_key = YOUR_AWS_SECRET_ACCESS_KEY

Make sample put app

# emacs put.php
<?php
require 'vendor/autoload.php';
use Aws\DynamoDb\DynamoDbClient;
$client = DynamoDbClient::factory(array(
    'profile' => 'default',
    'region' => 'ap-northeast-1',
    'version' => '2012-08-10',
    'credentials.cache' => true,
    'validation' => false,
    'scheme' => 'http'
));
$response = $client->putItem(array(
    'TableName' => 'ProductCatalog',
    'Item' => array(
        'Id'       => array('S'      => '104'      ), // Primary Key
        'Title'    => array('S'      => 'Book 104 Title' ),
        'ISBN'     => array('S'      => '111-1111111111' ),
        'Price'    => array('N'      => '25' ),
        'Authors'  => array('SS'  => array('Author1', 'Author2') )
    )
));

Run.

# php put.php

Make sample get app

# emacs get.php
<?php
require 'vendor/autoload.php';
use Aws\DynamoDb\DynamoDbClient;
$client = DynamoDbClient::factory(array(
    'profile' => 'default',
    'region' => 'ap-northeast-1',
    'version' => '2012-08-10',
    'credentials.cache' => true,
    'validation' => false,
    'scheme' => 'http'
));
$response = $client->getItem(array(
    'TableName' => 'ProductCatalog',
    'Key' => array(
 'Id' => array( 'S' => '104' )
    )
));
print_r ($response['Item']);

Run

# php get.php
Array
(
    [ISBN] => Array
        (
            [S] => 111-1111111111
        )

    [Id] => Array
        (
            [S] => 104
        )

    [Authors] => Array
        (
            [SS] => Array
                (
                    [0] => Author1
                    [1] => Author2
                )

        )

    [Price] => Array
        (
            [N] => 25
        )

    [Title] => Array
        (
            [S] => Book 104 Title
        )

)

Benchmark
Setup Nginx.

# yum -y install nginx
# mkdir /var/lib/nginx/.aws/
# cp ~/.aws/credentials /var/lib/nginx/.aws/credentials
# cp -r ~/sample_app/* /usr/share/nginx/html/
# emacs /etc/php-fpm.d/www.conf
user = apache
group = apache
->
user = nginx
group = nginx

# emacs /etc/nginx/nginx.conf
    server {
        include /etc/nginx/default.d/*.conf;
 
# /etc/init.d/nginx start
# /etc/init.d/php-fpm start
# curl 127.0.0.1/get.php -v

Run benchmark.

# ab -c 100 -n 10000 http://127.0.0.1/put.php
Requests per second:    763.44 [#/sec] (mean)

# ab -c 100 -n 10000 http://127.0.0.1/get.php
Requests per second:    832.62 [#/sec] (mean)

Maybe. If I install AWS SDK with ClassPreloader, that will be more faster.
But, ClassPreloader does not work with AWS SDK for PHP version 3.
Please let me know, if you know how to use ClassPreloader.
Next section. I tried a other way.

Benchmark with Go DynamoDB Proxy
Use bbpd.
http://takeshiyako.blogspot.jp/2015/10/go-lang-bbpd-godynamo-http-proxy.html

Make sample apps.

# emacs /usr/share/nginx/html/get_go.php
<?php
$url = "http://localhost:12333/GetItemJSON";
$postdata = "{\"Key\": {\"Id\": {\"S\": \"104\"}}, \"TableName\": \"ProductCatalog\"}";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
$result = curl_exec($ch);
curl_close($ch);
echo $result;

# emacs /usr/share/nginx/html/put_go.php
<?php
$random = substr(str_shuffle('1234567890abcdefghijklmnopqrstuvwxyz'), 0, 36);
$url = "http://localhost:12333/PutItemJSON";
$postdata = "{\"Item\": {\"Id\": \"$random\",\"Title\":\"Book 107 Title\",\"ISBN\":\"111-1111111111\",\"Price\":\"33\",\"Authors\":[\"Author1\", \"Author2\"]},\"TableName\": \"ProductCatalog\"}";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
$result = curl_exec($ch);
curl_close($ch);
echo $result;

Run benchmark.

# ab -c 100 -n 10000 http://127.0.0.1/get_go.php
Requests per second:    4260.64 [#/sec] (mean)

# ab -c 100 -n 10000 http://127.0.0.1/put_go.php
Requests per second:    3753.05 [#/sec] (mean)

It seems this solution is better than AWS SDK for PHP.
NOTE: When update same Id, many RETRY occurs with authreq. result of benchmark is 750 req/s.
if you update many data, please carefully.
https://github.com/smugmug/godynamo/blob/master/authreq/authreq.go#L91

See also
AWS SDK for PHP 3.x
http://docs.aws.amazon.com/aws-sdk-php/v3/api/index.html
Requirements — AWS SDK for PHP documentation
http://docs.aws.amazon.com/aws-sdk-php/v3/guide/getting-started/requirements.html
Performance Guide — AWS SDK for PHP 2.8.22 documentation
http://docs.aws.amazon.com/aws-sdk-php/v2/guide/performance.html
Top image from Felix De Vliegher Enjoying the view

Go lang hot-deploy with go-server-starter

https://github.com/lestrrat/go-server-starter

The start_server utility is a superdaemon for hot-deploying server programs.

Let's try.
Environment: EC2 AMI ID: Amazon Linux AMI 2015.09 (HVM), SSD Volume Type - ami-9a2fb89a

Go Sample app
Install Go

$ sudo yum -y install golang
$ go version
go version go1.4.2 linux/amd64
$ mkdir ~/gocode
$ echo 'export GOPATH=$HOME/gocode' >> ~/.bashrc
$ source $HOME/.bashrc

Install go-server-starter

# go get github.com/lestrrat/go-server-starter/cmd/start_server
# $GOPATH/bin/start_server --version
0.0.2
# $GOPATH/bin/start_server --help

Make sample web app

# mkdir $GOPATH/src/web/
# cd $GOPATH/src/web/
# emacs sample.go
package main

import (
 "fmt"
 "github.com/lestrrat/go-server-starter/listener"
 "log"
 "net"
 "net/http"
 "os"
)

func hello(w http.ResponseWriter, r *http.Request) {
 fmt.Fprintf(w, "Hello world!")
}

func newHandler() http.Handler {
 mux := http.NewServeMux()
 mux.HandleFunc("/", hello)
 return mux
}

func main() {
 var l net.Listener

 // check environment variable
 if os.Getenv("SERVER_STARTER_PORT") != "" {
  listeners, err := listener.ListenAll()
  if err != nil {
   fmt.Println(err)
   return
  }

  if 0 < len(listeners) {
   l = listeners[0]
  }
 }

 // default port
 if l == nil {
  var err error
  l, err = net.Listen("tcp", fmt.Sprintf(":8080"))

  if err != nil {
   fmt.Println(err)
   return
  }
 }

 log.Printf("Start Go HTTP Server")
 fmt.Println(http.Serve(l, newHandler()))

}

# go install
# $GOPATH/bin/web
2015/10/26 07:36:23 Start Go HTTP Server
# curl localhost:8080
Hello world!

Start app with start_server

# KILL_OLD_DELAY=5 nohup $GOPATH/bin/start_server --port=8000 --pid-file=$GOPATH/start_server.pid -- $GOPATH/bin/web >> $GOPATH/start_server.log &

# cat $GOPATH/start_server.log
starting new worker 26499
2015/10/26 08:36:55 Start Go HTTP Server

$ curl http://127.0.0.1:8000/
Hello world!

Try to hot-deploy.
# cd $GOPATH/src/web/
# emacs sample.go
        fmt.Fprintf(w, "Hello world!")
->
        fmt.Fprintf(w, "Hello world!!!!")
# go install

Graceful restart.

# /bin/cat $GOPATH/start_server.pid | xargs kill -HUP

Check log.

# cat $GOPATH/start_server.log
received HUP (num_old_workers=TODO)
spawning a new worker (num_old_workers=TODO)
starting new worker 2618
2015/10/27 02:43:21 Start Go HTTP Server
new worker is now running, sending TERM to old workers:2576
sleep 5 secs
killing old workers
old worker 2576 died, status:512

Check result.

$ curl http://127.0.0.1:8000/
Hello world!!!!

That's fine.

No error with graceful restart?
Set Nginx for Reverse Proxy
Nginx is necessary for safe connection from the HTTP cliant. Nginx Reverse Proxy makes re-handshake for the backend.

$ sudo yum -y install nginx
$ sudo emacs /etc/nginx/nginx.conf
worker_processes  1;
events {
    worker_connections  50000;
}
http {
    include     mime.types;
    access_log  off;
    upstream app {
        server 127.0.0.1:8000 max_fails=10;
        server localhost:8000 max_fails=10 backup;
    }
    server {
        location / {
            proxy_pass http://app;
            proxy_next_upstream error http_502;
        }
    }
}
$ sudo service nginx start
$ curl http://127.0.0.1/
Hello world!!!!

Run long time ab

$ sudo yum -y install httpd-tools
# ab -c 10 -n 200000 http://127.0.0.1/

Graceful restart

$ /bin/cat $GOPATH/start_server.pid | xargs kill -HUP
$ tail -f  $GOPATH/start_server.log
received HUP (num_old_workers=TODO)
spawning a new worker (num_old_workers=TODO)
starting new worker 2766
2015/10/27 02:56:48 Start Go HTTP Server
new worker is now running, sending TERM to old workers:2759
sleep 5 secs
killing old workers
old worker 2759 died, status:512

Result from ab

Failed requests:        0

Benchmark
start_server support UNIX domain socket. Let's Benchmark Port vs UNIX domain socket.
Instance type: c4.xlarge
Benchmark command.

# ab -c 100 -n 1000000 http://127.0.0.1/

start_server start commands:

Port
# KILL_OLD_DELAY=5 nohup $GOPATH/bin/start_server --port=8000 --pid-file=$GOPATH/start_server.pid -- $GOPATH/bin/web >> $GOPATH/start_server.log &

UNIX domain socket
$ KILL_OLD_DELAY=5 nohup $GOPATH/bin/start_server --path /tmp/app.sock --pid-file=$GOPATH/start_server.pid -- $GOPATH/bin/web >> $GOPATH/start_server.log &
$ chmod 777 /tmp/app.sock

Nginx confs:

Port
    upstream app {
        server 127.0.0.1:8000;
    }

UNIX domain socket
    upstream app {
        server unix:/tmp/app.sock;
    }

Results.

Port
Requests per second:    12544.89 [#/sec] (mean)

UNIX domain socket
Requests per second:    18435.59 [#/sec] (mean)

Tips
How to stop start_server

# /bin/cat $GOPATH/start_server.pid | xargs kill -TERM

# cat $GOPATH/start_server.log
received TERM, sending TERM to all workers:2766
worker 2766 died, status:512
exiting

See also.
Server Starter - a superdaemon to hot-deploy server programs
http://www.slideshare.net/kazuho/server-starter-a-superdaemon-to-hotdeploy-server-programs
Fix panic when SERVER_STARTER_PORT is not defined by handlename · Pull Request #3 · lestrrat/go-server-starter
https://github.com/lestrrat/go-server-starter/pull/3
Docker と SO_REUSEPORT を組み合わせてコンテナのHot Deployにチャレンジ - blog.nomadscafe.jp
http://blog.nomadscafe.jp/2015/01/docker-and-so-reuseport-hot-deploy.html
Server::Starterから学ぶhot deployの仕組み - $shibayu36->blog;
http://blog.shibayu36.org/entry/2012/05/07/201556
Top image from John Lord Starter

Go lang bbpd is GoDynamo HTTP Proxy

https://github.com/smugmug/bbpd

bbpd is a http proxy server for Amazon's DynamoDB service.

bbpd uses GoDynamo package for the Amazon DynamoDB database. It made by smugmug.
It supports JSON documents and can post/get data through from HTTP POST method. Oh, that is pretty nice.
So let's try to make DynamoDB HTTP Proxy.

Environment: EC2 AMI ID: Amazon Linux AMI 2015.09 (HVM), SSD Volume Type - ami-9a2fb89a

Install Go

$ sudo yum -y install golang zsh
$ go version
go version go1.4.2 linux/amd64
$ mkdir ~/gocode
$ echo 'export GOPATH=$HOME/gocode' >> ~/.bashrc
$ source $HOME/.bashrc

Install bbpd

$ go get github.com/smugmug/bbpd
$ sudo cp $GOPATH/bin/bbpd /usr/bin/
$ sudo cp $GOPATH/src/github.com/smugmug/bbpd/bin/bbpd/bbpd_ctl /usr/bin/
$ sudo cp $GOPATH/src/github.com/smugmug/bbpd/bin/bbpd/bbpd_daemon /usr/bin/
$ sudo chmod 755 /usr/bin/bbpd_ctl
$ sudo chmod 755 /usr/bin/bbpd_daemon

Set config file.

$ cp $GOPATH/src/github.com/smugmug/godynamo/conf_file/test_aws-config.json ~/.aws-config.json
$ emacs ~/.aws-config.json
{
    "extends":[],
    "services": {
        "default_settings":{
            "params":{
                "access_key_id":"myAccessKey",
                "secret_access_key":"mySecret",
                "use_sys_log":true
            }
        },
        "dynamo_db": {
            "host":"dynamodb.us-east-1.amazonaws.com",
            "zone":"us-east-1",
            "scheme":"http",
            "port":80,
            "keepalive":false,
            "iam": {
                "use_iam":false,
                "role_provider":"",
                "access_key":"",
                "secret_key":"",
                "token":"",
                "base_dir":"",
                "watch":false
            }
        }
    }
}

Start.

$ bbpd_ctl start
**** starting bbpd as daemon
bbpd - started
2015/10/23 02:22:17 global conf.Vals initialized
2015/10/23 02:22:17 not using iam, assume credentials hardcoded in conf file
2015/10/23 02:22:17 starting bbpd...
2015/10/23 02:22:17 induce panic with ctrl-c (kill -2 2801) or graceful termination with kill -[1,3,15] 2801
2015/10/23 02:22:17 trying to bind to port:12333
2015/10/23 02:22:17 init routing on port 12333
2015/10/23 02:22:17 global conf.Vals initialized
2015/10/23 02:22:17 not using iam, assume credentials hardcoded in conf file
2015/10/23 02:22:17 starting bbpd...
2015/10/23 02:22:17 induce panic with ctrl-c (kill -2 2804) or graceful termination with kill -[1,3,15] 2804
2015/10/23 02:22:17 trying to bind to port:12333
2015/10/23 02:22:17 port 12333 already in use
2015/10/23 02:22:17 trying to bind to port:12334
2015/10/23 02:22:17 init routing on port 12334

Check processes.

$ pgrep -l bbpd
24330 bbpd
24333 bbpd

Get status.

$ curl -H "X-Bbpd-Indent: true" "http://localhost:12333/Status"
{
 "Status": "ready",
 "AvailableHandlers": [
  "/DescribeTable/",
  "/DeleteItem",
  "/ListTables",
  "/CreateTable",
  "/UpdateTable",
  "/StatusTable/",
  "/PutItem",
  "/PutItemJSON",
  "/GetItem",
  "/GetItemJSON",
  "/BatchGetItem",
  "/BatchGetItemJSON",
  "/BatchWriteItem",
  "/BatchWriteItemJSON",
  "/UpdateItem",
  "/Query",
  "/Scan",
  "/RawPost/",
  "/"
 ],
 "Args": {
  "X-Bbpd-Indent": "set '-H \"X-Bbpd-Indent: True\" ' to indent the top-level json",
  "X-Bbpd-Verbose": "set '-H \"X-Bbpd-Verbose: True\" ' to get verbose output"
 },
 "Summary": {
  "StartTime": "2015-10-23 08:12:18.875147234 +0000 UTC",
  "RunningTime": "11.531750029s",
  "LongestResponse": "0.00ms",
  "AverageResponse": "0.00ms",
  "LastResponse": "no requests made yet",
  "ResponseCount": "0"
 }
}

Now, bbpd is ready.

Run curl
Sample scripts are here. https://github.com/smugmug/bbpd/tree/master/tests

Create table

$ curl -H "X-Amz-Target: DynamoDB_20120810.CreateTable" -X POST -d '
{
    "AttributeDefinitions": [
        {
            "AttributeName": "user_id", 
            "AttributeType": "S"
        }
    ], 
    "KeySchema": [
        {
            "AttributeName": "user_id", 
            "KeyType": "HASH"
        }
    ], 
    "ProvisionedThroughput": {
        "ReadCapacityUnits": 100, 
        "WriteCapacityUnits": 100
    }, 
    "TableName": "bbpd-test"
}
' http://localhost:12333/

Put

curl -H "X-Amz-Target: DynamoDB_20120810.PutItem"  -X POST -d '
{
    "Item": {
        "num": 1, 
        "numlist": [
            6, 
            7, 
            1, 
            2, 
            3, 
            9, 
            -7234234234.234234
        ], 
        "stringlist": [
            "pk1_a", 
            "pk1_b", 
            "pk1_c"
        ], 
        "user_id": "my_user_id1"
    }, 
    "TableName": "bbpd-test"
}
' http://localhost:12333/PutItemJSON

Get

$ curl -H "X-Amz-Target: DynamoDB_20120810.GetItem" -X POST -d '
{
    "Key": {
        "user_id": {
            "S": "my_user_id1"
        }
    }, 
    "TableName": "bbpd-test"
}
' "http://localhost:12333/GetItemJSON"

{"Item":{"num":1,"numlist":[-7.234234234234234e+09,3,9,7,2,1],"stringlist":["pk1_c","pk1_a","pk1_b"],"user_id":"my_user_id1"}}

Looks fine!

Check table by SQL using ddbcli
https://github.com/winebarrel/ddbcli

$ gem install ddbcli --no-ri --no-rdoc
Fetching: ddbcli-0.5.3.gem (100%)

$ ddbcli -k ACCESS_KEY -s SECRET_KEY -r REGION_OR_ENDPOINT
ap-northeast-1> select all * from bbpd-test;
[
  {"num":1,"numlist":[6,3,9,7,2,1,-7234234234.234234],"stringlist":["pk1_a","pk1_b","pk1_c"],"user_id":"my_user_id1"}
]
// 1 row in set (0.02 sec)

That's correct.

Benchmark

DynamoDB
Provisioned Read Capacity Units: 10000
Provisioned Write Capacity Units: 10000

EC2
Instance type: c4.large, c4.xlarge, c4.2xlarge

# emacs postdata.file
{
    "Key": {
        "user_id": {
            "S": "my_user_id1"
        }
    },
    "TableName": "bbpd-test"
}

# ab -n 100000 -c 100 -p postdata.file -T "application/json" http://localhost:12333/GetItemJSON

c4.large
Requests per second:    4754.25 [#/sec] (mean)

c4.xlarge
Requests per second:    8591.35 [#/sec] (mean)

c4.2xlarge
Requests per second:    8540.63 [#/sec] (mean) 
Many RETRY occurs with godynamo authreq.retryReq.

Tips
Set log dir.

$ sudo emacs /usr/bin/bbpd_ctl

LOG=/path/to/log/bbpd.log

    start)
        echo -n "**** starting bbpd as daemon\n"
        $INSTALL_PATH/$DAEMON >> $LOG 2>&1 || true
        $INSTALL_PATH/$DAEMON >> $LOG 2>&1 || true
        echo "bbpd - started\n"
        ;;

Amazon DynamoDB logo / Amazon Web Services LLC - http://aws.typepad.com/aws/2011/12/introducing-aws-simple-icons-for-your-architecture-diagrams.html
gopher-side_color.{ai,svg,png} was created by Takuya Ueda (http://u.hinoichi.net). Licensed under the Creative Commons 3.0 Attributions license. - https://github.com/golang-samples/gopher-vector#gopher-side_color
The Go gopher was designed by Renee French. (http://reneefrench.blogspot.com/)

Go lang Goji + Einhorn provides "graceful restart"

Goji aka WAF of Golang.
Says.

Automatic support for Einhorn, systemd, and more
Graceful shutdown, and zero-downtime graceful reload when combined with Einhorn.

Looks very pritty.
So, Let's try Goji, and graceful restart by Einhorn.

Make Goji app.
Get Goji.

$ go get github.com/zenazn/goji

Make sample goji app.

$ mkdir ~/gocode/src/goji_sample/
$ cd ~/gocode/src/goji_sample/
$ emacs sample1.go
package main

import (
 "fmt"
 "net/http"

 "github.com/zenazn/goji"
 "github.com/zenazn/goji/web"
)

func hello(c web.C, w http.ResponseWriter, r *http.Request) {
 fmt.Fprintf(w, "Hello?, %s!", c.URLParams["name"])
}

func main() {
 goji.Get("/hello/:name", hello)
 goji.Serve()
}

$ go install

Set up Einhorn and run app.
Install Einhorn with gem.

$ sudo gem install einhorn

Start Goji app with Einhorn.

$ nohup einhorn -b 0.0.0.0:8001 -c einhorn_with_goji_sample -m manual $GOPATH/bin/goji_sample >> /tmp/app.log&

It works?
Chak HTTP result via curl.

$ curl 0.0.0.0:8001/hello/aaa
Hello?, aaa!

It warks!
Check app log.

$ tail -f /tmp/app.log
[MASTER 26118] INFO: Writing PID to /var/folders/6_/n62p97qx08x53ncc8hqn5fvm0000gp/T/einhorn-einhorn_with_sample_goji.pid
[MASTER 26118] INFO: Binding to 0.0.0.0:8001 with flags []
[MASTER 26118] INFO: Launching 1 new workers
[MASTER 26118] INFO: ===> Launched 26122 (index: 0)
[WORKER 26122] INFO: About to exec ["/Users/yako/gocode/bin/goji_sample"]
2015/10/21 18:11:35.258055 Starting Goji on 0.0.0.0:8001
2015/10/21 18:11:35.258153 bind: ACKing to einhorn
[MASTER 26118] INFO: [client 2:11] client connected
[MASTER 26118] INFO: Received a manual ACK from 26122
[MASTER 26118] INFO: Up to 1 / 1 manual ACKs
[MASTER 26118] INFO: [client 2:11] client disconnected
2015/10/21 18:11:49.985358 [ore-no-MacBook-2.local/zVchqfLPzY-000001] Started GET "/hello/aaa" from 127.0.0.1:61537
2015/10/21 18:11:49.986692 [ore-no-MacBook-2.local/zVchqfLPzY-000001] Returning 200 in 367.506µs

Check process.

$ ps | grep goji
26197 ttys000    0:00.22 einhorn: ruby einhorn_with_goji_sample
26201 ttys000    0:00.01 /Users/yako/gocode/bin/goji_sample

Graceful restart Goji with Einhorn
Modify current sample app.

$ emacs sample1.go
        fmt.Fprintf(w, "Hello?, %s!", c.URLParams["name"])
->
        fmt.Fprintf(w, "Hello!!!, %s!", c.URLParams["name"])

$ go install

Run Einhorn upgrade command. this command do graceful restart Goji app.

$ einhornsh -c einhorn_with_goji_sample -e upgrade
Welcome, yako! You are speaking to Einhorn Master Process 26197 (einhorn_with_goji_sample).
This is Einhorn 0.6.4.
Upgrading smoothly, as commanded
Starting smooth upgrade from version 0...
===> Launched 26273 (index: 1)
Upgraded successfully to version 1 (Einhorn 0.6.4).
Upgrade done

$ curl 0.0.0.0:8001/hello/aaa
Hello!!!, aaa!

No error with graceful restart?
Run long time ab(Apache Bench) and then run graceful result.

$ ab -c 1 -n 10000 http://0.0.0.0:8001/hello/aaa

$ einhornsh -c einhorn_with_goji_sample -e upgrade

Chcek result of ab.

Failed requests:        0

looks fine.

Tips
How to shotdown Einhorn worker.

$  einhornsh -c einhorn_with_goji_sample -e die
Welcome, yako! You are speaking to Einhorn Master Process 26529 (einhorn_with_goji_sample).
This is Einhorn 0.6.4.
Einhorn is going down! Successfully sent USR2s to 1 processes: [26586]

$ tail -f /tmp/app.log
[MASTER 26529] INFO: [client 4:11] client connected
[MASTER 26529] INFO: Sending USR2 to [26586]
2015/10/21 19:24:57.165737 Goji received signal, gracefully stopping
2015/10/21 19:24:57.165790 Goji stopped
[MASTER 26529] INFO: ===> Exited worker 26586
[MASTER 26529] INFO: [client 4:11] client disconnected

See also.
Build and Deploy to production · Issue #105 · zenazn/goji
https://github.com/zenazn/goji/issues/105
Meet Einhorn
https://stripe.com/blog/meet-einhorn
lestrrat/go-server-starter Go port of start_server utility (Server::Starter)
https://github.com/lestrrat/go-server-starter
fvbock/endless Zero downtime restarts for golang HTTP and HTTPS servers.
https://github.com/fvbock/endless
guregu/kami web "framework" using x/net/context
https://github.com/guregu/kami
facebookgo/grace Graceful restart for Go servers.
https://github.com/facebookgo/grace
dcu/http-einhorn
https://github.com/dcu/http-einhorn
Top image form Panagiotis Giannakopoulos

Go lang Syntax highlighting with Emacs Mac OS X

This example provides Syntax highlighting support for Google Go Language (golang) in Emacs.
I use El-Get with Emacs.

Syntax highlighting
Add El-Get code to Emacs int.el file.

$ emacs ~/.emacs.d/init.el
(when load-file-name
  (setq user-emacs-directory (file-name-directory load-file-name)))

(add-to-list 'load-path (locate-user-emacs-file "el-get/el-get"))
(unless (require 'el-get nil 'noerror)
  (with-current-buffer
      (url-retrieve-synchronously
       "https://raw.githubusercontent.com/dimitri/el-get/master/el-get-install.el")
    (goto-char (point-max))
    (eval-print-last-sexp)))
    ;; golang
    (el-get-bundle go-mode)

Open .go file.

$ emacs sample.go

Thats it.

goimports
https://godoc.org/golang.org/x/tools/cmd/goimports

Command goimports updates your Go import lines, adding missing ones and removing unreferenced ones.

Lets set goimports..
Add GOPATH and PATH to .bashrc. GOPATH is ~/gocode.

$ emacs ~/.bashrc
export GOPATH=$HOME/gocode
 if [ -d $GOPATH/bin ]; then
     PATH=$PATH:$GOPATH/bin
 fi
$ source $HOME/.bashrc

Get goimports.

$ go get golang.org/x/tools/cmd/goimports

Set ~/.emacs.d/init.el.

$ emacs ~/.emacs.d/init.el
(when load-file-name
  (setq user-emacs-directory (file-name-directory load-file-name)))

(add-to-list 'load-path (locate-user-emacs-file "el-get/el-get"))
(unless (require 'el-get nil 'noerror)
  (with-current-buffer
      (url-retrieve-synchronously
       "https://raw.githubusercontent.com/dimitri/el-get/master/el-get-install.el")
    (goto-char (point-max))
    (eval-print-last-sexp)))
    (el-get-bundle go-mode)

(add-hook 'before-save-hook 'gofmt-before-save)
(let ((goimports (executable-find "goimports")))
  (if goimports (setq gofmt-command goimports)))

Lets make new sample code. this code has no import and no Indentation for exsample.

$ mkdir $GOPATH/src/sample
$ emacs $GOPATH/src/sample/sample_code.go
package main

func main() {
fmt.Printf(stringutil.Reverse("!oG ,olleH"))
}

Run Emacs save command.

C-x C-s

package main

import "fmt"

func main() {
        fmt.Printf(stringutil.Reverse("!oG ,olleH"))
}

The new line of import was added automaticaly. Nice!

Go langでrandを使って擬似乱数を作る

Go langで擬似乱数を作ってみます。
簡単なコードとしては以下のようになります。
0から9までの数字を10回出力します。

package main

import (
    "fmt"
    "math/rand"
    "time"
)

func main() {
    rand.Seed(time.Now().UnixNano())
    for i := 0; i < 10; i++ {
        fmt.Println("My favorite number is", rand.Intn(10))
    }
}

$ go run rand.go
My favorite number is 0
My favorite number is 7
My favorite number is 0
My favorite number is 8
My favorite number is 3
My favorite number is 3
My favorite number is 7
My favorite number is 5
My favorite number is 0
My favorite number is 8

rand.Seed(time.Now().UnixNano()) という箇所で乱数の元になるシードを与えるのがポイントです。
なぜ、シードを与えるかというと、ここで使っている関数rand.Intnは、randパッケージに新しい乱数ジェネレータ生成するときに同じ初期化 rand.NewSource(1) をしているそうで、シードをあたえて、異なる乱数を出す必用があるからだそうです。

以下、シードを与えていないと、同じ結果になる例です。

func main() {
    // シードを与えないで実行
    for i := 0; i < 10; i++ {
        fmt.Print(rand.Intn(10), " ")
    }
    fmt.Println(" ")

    // randをNewSourceで初期化して実行
    my_rand := rand.New(rand.NewSource(1))
    for i := 0; i < 10; i++ {
        fmt.Print(my_rand.Intn(10), " ")
    }
    fmt.Println(" ")

    // シードを与えて実行
    my_rand.Seed(time.Now().UnixNano())
    for i := 0; i < 10; i++ {
        fmt.Print(my_rand.Intn(10), " ")
    }
}

$ go run rand.go
1 7 7 9 1 8 5 0 6 0
1 7 7 9 1 8 5 0 6 0
8 7 1 0 1 4 9 0 3 0
$ go run rand.go
1 7 7 9 1 8 5 0 6 0
1 7 7 9 1 8 5 0 6 0
4 0 4 7 2 0 2 9 7 0

乱数を使って抽選を実装したlotteryという面白いライブラリを作っている方がいたので、遊んでみました。
go get実行を実行してgithubのコードを取得。

$ go get github.com/kyokomi/lottery
$ tree ~/gocode/src/github.com/
/Users/yako/gocode/src/github.com/
└── kyokomi
    └── lottery
        ├── LICENSE
        ├── README.md
        ├── circle.yml
        ├── lottery.go
        └── lottery_test.go

2 directories, 5 files

1/5の確率でアタリ、それ以外をハズレとして、1000回出力。アタリが出た確率も最後に出力します。

package main

import (
        "fmt"
        "github.com/kyokomi/lottery"
        "math/rand"
        "time"
)

func main() {
        lot := lottery.New(rand.New(rand.NewSource(time.Now().UnixNano())))

        check := 1000
        count := 0

        for i := 0; i < check; i++ {

                if lot.LotOf(1, 5) {
                        // 1/5の確率
                        fmt.Print("アタリ")
                        count++
                } else {
                        fmt.Print("ハズレ")
                }

        }

        fmt.Println(float32(count)/float32(check)*100, "%")

}

$ go run lottery.go
ハズレハズレハズレアタリアタリアタリハズレハズレハズレアタリハズレアタリハズレハズレハズレハズレハズレハズレハズレハズレアタリハズレハズレハズレハズレハズレアタリハズレハズレハズレハズレハズレハズレハズレハズレハズレアタリハズレハズレハズレハズレハズレハズレアタリハズレアタリハズレハズレハズレハズレアタリハズレハズレアタリハズレアタリハズレアタリハズレハズレアタリアタリハズレアタリハズレハズレアタリハズレハズレハズレハズレハズレハズレハズレハズレハズレアタリハズレハズレハズレハズレアタリハズレハズレハズレハズレハズレアタリアタリハズレハズレハズレアタリハズレハズレアタリアタリハズレハズレハズレハズレハズレハズレハズレアタリハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレアタリハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレアタリハズレアタリハズレハズレハズレアタリハズレハズレハズレハズレハズレアタリアタリハズレアタリハズレハズレハズレハズレハズレハズレハズレハズレハズレハズレアタリアタリハズレアタリハズレハズレハズレハズレハズレハズレアタリアタリハズレハズレハズレハズレアタリハズレハズレアタリ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
19.9 %

抽選プログラムを作る機会があったら使ってみるとよさそうです。

参考文献）
rand - The Go Programming Language
https://golang.org/pkg/math/rand/
お気楽 Go 言語プログラミング入門 並列プログラミング

http://www.geocities.jp/m_hiroi/golang/abcgo14.html

Goでlotteryという抽選ライブラリを作った - きょこみのーと
http://kyokomi.hatenablog.com/entry/2015/07/26/151138

Top image from Anthony Starks Gopher inspecting Go code
The Go gopher was designed by Renee French. http://reneefrench.blogspot.com/

Go langのnet/httpでHTTPサーバーを立てる

Go langは、簡単にウェブサーバを立てられるという話を聞いたので、ウェブサーバを立ててみます。
ここでは、Go langのhttpパッケージを使いました。

Hello, World
まずは、簡単に"Hello, World"を返すサーバです。

package main

import (
    "fmt"
    "net/http"
    "log"
)

// http.HandleFuncに登録する関数
// http.ResponseWriterとhttp.Requestを受ける
func HelloServer(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hello, World\n")
}

func TestServer(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "TEST\n")
}

func main() {
    // http.HandleFuncにルーティングと処理する関数を登録
    http.HandleFunc("/", HelloServer) 
    http.HandleFunc("/test", TestServer)

    // ログ出力
    log.Printf("Start Go HTTP Server")

    // http.ListenAndServeで待ち受けるportを指定
    err := http.ListenAndServe(":4000", nil)

    // エラー処理
    if err != nil {
       log.Fatal("ListenAndServe: ", err)
    }
}

go run

サーバスタート
$ go run web.go
2015/10/19 16:55:08 Start Go HTTP Server

HTTPレスポンスを確認
$ curl localhost:4000
Hello, World
$ curl localhost:4000/test
TEST

同じサーバをスタートすると。既にポートを使っているのでエラー。
$ go run web.go
2015/10/19 16:55:18 Start Go HTTP Server
2015/10/19 16:55:18 ListenAndServe: listen tcp :4000: bind: address already in use
exit status 1

html/template
GoのHTMLテンプレートにはメジャーどころとして text/template と html/template があるらしいです。
ここでは、html/templateパッケージを利用して、テンプレートファイルを読み込んで出力します。こちらの方がセキュアなHTMLを生成するそうです。
テンプレートは、親テンプレートと小テンプレートの２つを用意しました。
現在時刻を表示する単純なウェブページを作ります。
時刻オブジェクトを文字列に変換するときは、fmt.Sprint(hoo)を使いました。

文字列化するの、細かい事言わないならだいたい fmt.Sprint(foo) でなんとかなります。 / “Golangでの文字列・数値変換 - 小野マトペの納豆ペペロンチーノ日記” http://t.co/zYxAW7xmvq

— mattn (@mattn_jp) April 22, 2014

ほかにも、文字・数値変換にはstrconvパッケージも使えます。この場合は、パッケージをインポートして、unixtime := strconv.FormatInt(time.Now().Unix(), 10)というふうに書きます。これはどちらかと言うと、２進数表現にしたい時など、細かい用途に使うのかな？と思いました。
web.go

package main

import (
    "fmt"
    "net/http"
    "log"
    "html/template"
    "time"
)

type TemplateData struct {
    Title string
    Datetime string
    Unixtime string
}

func HelloServer(w http.ResponseWriter, r *http.Request) {
    // 親テンプレートindex.htmlと、小テンプレートbody.htmlを用意
    tmpl := template.Must(template.ParseFiles("views/index.html", "views/body.html"))

    // タイトル
    title := "現在の時刻"

    // bodyに表示するコンテンツ
    // 時刻オブジェクトを文字列に変換
    datetime := fmt.Sprint(time.Now())
    unixtime := fmt.Sprint(time.Now().Unix())

    // テンプレートを実行して出力
    templatedata := TemplateData{title, datetime, unixtime}
    if err := tmpl.ExecuteTemplate(w, "base", templatedata); err != nil {
        fmt.Println(err)
    }
}

func main() {
    http.HandleFunc("/", HelloServer)
    log.Printf("Start Go HTTP Server")
    err := http.ListenAndServe(":4000", nil)
    if err != nil {
       log.Fatal("ListenAndServe: ", err)
    }
}

views/index.html

{{define "base"}}
<html>
    <head>
        <meta charset="utf-8">
        <title>{{.Title}}</title>
    </head>
    <body>
        {{template "body" .}}
    </body>
</html>
{{end}}

views/body.html

{{define "body"}}
    <dev id="content">
        <p>Datetime: {{.Datetime}}</p>
        <p>Unixtime: {{.Unixtime}}</p>
    </dev>
{{end}}

go run

$ go run web.go
2015/10/20 09:57:18 Start Go HTTP Server
$ curl localhost:4000
<html>
    <head>
        <meta charset="utf-8">
        <title>現在の時刻</title>
    </head>
    <body>

    <dev id="content">
        <p>Datetime: 2015-10-20 09:57:19.297331721 &#43;0900 JST</p>
        <p>Unixtime: 1445302639</p>
    </dev>

    </body>
</html>

なにか色々できそうな気がしてきますね。


参考文献）
http - The Go Programming Language
https://golang.org/pkg/net/http/
A Tour of Go Web servers
http://go-tour-jp.appspot.com/#57
Go言語によるwebアプリの作り方
http://www.slideshare.net/yasi_life/goweb-16448500
Codelab: Webアプリケーションを書いてみよう - golang.jp
http://golang.jp/codelab-wiki
テンプレートの処理 | build-web-application-with-golang
https://astaxie.gitbooks.io/build-web-application-with-golang/content/ja/07.4.html
逆引きGolang (日付と時刻)
http://ashitani.jp/golangtips/tips_time.html
Golangでの文字列・数値変換 - 小野マトペの納豆ペペロンチーノ日記
http://matope.hatenablog.com/entry/2014/04/22/101127

Title image from Alyson Hurt
The Go gopher was designed by Renee French. http://reneefrench.blogspot.com/

Go lang "Hello, world" on Mac OS X

This is a sample Go app for "Hello, world".

1) get "Apple OS X" install package from download page.
https://golang.org/dl/

2) Install gox.x.x.darwin-amd64.pkg on your Mac.

3) make code dir and set $GOPATH

$ mkdir ~/gocode
$ echo 'export GOPATH=$HOME/gocode' >> ~/.bashrc
$ source $HOME/.bashrc

4) make "Hello, world."

$ mkdir -p ~/gocode/src/hello
$ cd ~/gocode/src/hello
$ emacs hello.go
package main

import "fmt"

func main() {
 fmt.Printf("Hello, world.\n")
}

$ go install
$ tree ~/gocode
/Users/yako/gocode
├── bin
│   └── hello
└── src
    └── hello
        └── hello.go

3 directories, 2 files

$ $GOPATH/bin/hello
Hello, world.

5) make library.

$ mkdir ~/gocode/src/stringutil
$ cd ~/gocode/src/stringutil
$ emacs reverse.go
// Package stringutil contains utility functions for working with strings.
package stringutil

// Reverse returns its argument string reversed rune-wise left to right.
func Reverse(s string) string {
 r := []rune(s)
 for i, j := 0, len(r)-1; i < len(r)/2; i, j = i+1, j-1 {
  r[i], r[j] = r[j], r[i]
 }
 return string(r)
}

$ cd ~/gocode/src/hello
$ emacs hello.go
package main

import (
       "fmt"

       "stringutil"
)

func main() {
     fmt.Printf(stringutil.Reverse("!oG ,olleH"))
}

$ go install
$ tree ~/gocode
/Users/yako/gocode
├── bin
│   └── hello
├── pkg
│   └── darwin_amd64
│       └── stringutil.a
└── src
    ├── hello
    │   ├── hello.go
    └── stringutil
        └── reverse.go

6 directories, 5 files

$ $GOPATH/bin/hello
Hello, Go!

6) make reverse test.

$ cd ~/gocode/src/stringutil
$ emacs reverse_test.go
package stringutil

import "testing"

func TestReverse(t *testing.T) {
 cases := []struct {
  in, want string
 }{
  {"Hello, world", "dlrow ,olleH"},
  {"Hello, 世界", "界世 ,olleH"},
  {"", ""},
 }
 for _, c := range cases {
  got := Reverse(c.in)
  if got != c.want {
   t.Errorf("Reverse(%q) == %q, want %q", c.in, got, c.want)
  }
 }
}

$ tree ~/gocode
/Users/yako/gocode
├── bin
│   └── hello
├── pkg
│   └── darwin_amd64
│       └── stringutil.a
└── src
    ├── hello
    │   └── hello.go
    └── stringutil
        ├── reverse.go
        └── reverse_test.go

6 directories, 5 files

$ go test stringutil
ok   stringutil 0.006s

See also.

How to Write Go Code
https://golang.org/doc/code.html

Top image from Yuko Honda Today's latte, celebrating Go version 1!
The Go gopher was designed by Renee French. http://reneefrench.blogspot.com/

PHP curl with CURLOPT_CONNECTTIMEOUT_MS

If you want to set timeout of milliseconds with PHP curl_setopt, you have to compile new cURL and re-install PHP.

http://php.net/manual/en/function.curl-setopt.php

CURLOPT_CONNECTTIMEOUT_MS
The number of milliseconds to wait while trying to connect. Use 0 to wait indefinitely. If libcurl is built to use the standard system name resolver, that portion of the connect will still use full-second resolution for timeouts with a minimum timeout allowed of one second.
Added in cURL 7.16.2. Available since PHP 5.2.3.

Environment: CentOS6 with PHP5.6

Install cURL

# cd /var/tmp
# git clone https://github.com/bagder/curl.git
# cd curl
# ./buildconf
# ./configure --enable-threaded-resolver
# make 
# make install
# ./src/curl -V
curl 7.45.0-DEV (x86_64-unknown-linux-gnu) libcurl/7.45.0-DEV OpenSSL/1.0.1e zlib/1.2.3 libidn/1.18
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile NTLM NTLM_WB SSL libz UnixSockets

Set ldconfig

# echo '/usr/local/lib' > /etc/ld.so.conf.d/custom-libs.conf
# ldconfig
# ldconfig -p | grep curl
 libcurl.so.4 (libc6,x86-64) => /usr/local/lib/libcurl.so.4
 libcurl.so.4 (libc6,x86-64) => /usr/lib64/libcurl.so.4
 libcurl.so (libc6,x86-64) => /usr/local/lib/libcurl.so
 libcurl.so (libc6,x86-64) => /usr/lib64/libcurl.so

Version up PHP

# yum update --skip-broken --disablerepo=base,updates --enablerepo=remi --enablerepo=remi-php56 php

Re-install php-common

# yum reinstall --skip-broken --disablerepo=base,updates --enablerepo=remi --enablerepo=remi-php56 php-common
# php -i | grep cURL
cURL support => enabled
cURL Information => 7.45.0-DEV

Make Test Script

# vim curl_test.php
<?php

$url = 'http://www.yahoo.co.jp/';
echo $url . "\n";

$option = array(
    CURLOPT_CONNECTTIMEOUT_MS => 500,
    CURLOPT_TIMEOUT           => 500,
    CURLOPT_HEADER            => FALSE,
    CURLOPT_FAILONERROR       => TRUE,
    CURLOPT_RETURNTRANSFER    => TRUE
);

if (!$ch = curl_init()) {
    error_log('[error] curl_init');
    exit;
}

try {
    if (!curl_setopt_array($ch, $option)) {
        throw new Exception('[error] curl_setopt');
    }

    if (!$result = curl_exec($ch)) {
        throw new Exception('[error] curl_exec');
    }

    curl_close($ch);

    echo $result;
} catch (Exception $e) {
    curl_close($ch);
    error_log($e->getMessage());
}

Run Test Script.

# php curl_test.php

Top pic from Francisco Gonzalez Snow Curl