install swift centos6.2

Install

対象環境

全て完璧そのままではないかもしれないが、以下の環境でなんとなく構築できる。

  • Amazon Linux AMI 2012.03 64bit
  • Centos 6.2 64bit
  • Ununtu 12.04 server 64bit
    (何かがうまくいかなかった気がするが思い出せない。。。

epelの追加(centos 6.2)

amazon linuxの場合既に追加済みなので、/etc/yum.repos.d/epel.repo を編集して、enabled=1とする。

CentOS 外部レポジトリの追加(EPEL)

sudo yum -y install yum-priorities

sudo vim /etc/yum.repos.d/CentOS-Base.repo

[base]
name=CentOS-$releasever - Base
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-5
priority=1
:
[updates]
priority=1
:
[addons]
priority=2
:
[extra]
priority=2
:
[centosplus]
enabled=1
priority=2
:
[contrib]
enabled=1
priority=2
:
wget http://ftp-srv2.kddilabs.jp/Linux/distributions/fedora/epel/6/x86_64/epel-release-6-5.noarch.rpm
sudo rpm -ivh epel-release-6-5.noarch.rpm

sudo vim /etc/yum.repos.d/epel.repo

[epel]
name=Extra Packages for Enterprise Linux 5 - $basearch
#baseurl=http://dl.fedoraproject.org/pub/epel/5/$basearch
mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=epel-5&arch=$basearch
failovermethod=priority
enabled=1
priority=2
:

install swift

SAIO – Swift All In One — Swift 1.4.9-dev documentation

rootで実行する。

sudo -i
yum install -y openstack-swift openstack-swift-proxy openstack-swift-account openstack-swift-container openstack-swift-object
yum install -y xinetd rsync
yum install -y memcached
yum install -y python-netifaces python-nose
yum install -y xfsprogs
yum install -y python-webob

Using a loopback device for storage

Swiftテスト用のディスクとしてファイルを作る。

mkdir /srv
dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1048576
mkfs.xfs -i size=1024 /srv/swift-disk
cp /etc/fstab /etc/fstab.bak
echo "/srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0" >> /etc/fstab
mkdir /mnt/sdb1
mount /mnt/sdb1
mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
chown swift:swift /mnt/sdb1/*
for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done
mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift
chown -R swift:swift /etc/swift /srv/[1-4]/ /var/run/swift

cp /etc/rc.local /etc/rc.local.bak
sed -i -e "s,^exit,mkdir /var/run/swiftnchown swift:swift /var/run/swiftnexit,g" /etc/rc.local
fi

echo "
uid = swift
gid = swift
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
address = 127.0.0.1

[account6012]
max connections = 25
path = /srv/1/node/
read only = false
lock file = /var/lock/account6012.lock

[account6022]
max connections = 25
path = /srv/2/node/
read only = false
lock file = /var/lock/account6022.lock

[account6032]
max connections = 25
path = /srv/3/node/
read only = false
lock file = /var/lock/account6032.lock

[account6042]
max connections = 25
path = /srv/4/node/
read only = false
lock file = /var/lock/account6042.lock

[container6011]
max connections = 25
path = /srv/1/node/
read only = false
lock file = /var/lock/container6011.lock

[container6021]
max connections = 25
path = /srv/2/node/
read only = false
lock file = /var/lock/container6021.lock

[container6031]
max connections = 25
path = /srv/3/node/
read only = false
lock file = /var/lock/container6031.lock

[container6041]
max connections = 25
path = /srv/4/node/
read only = false
lock file = /var/lock/container6041.lock

[object6010]
max connections = 25
path = /srv/1/node/
read only = false
lock file = /var/lock/object6010.lock

[object6020]
max connections = 25
path = /srv/2/node/
read only = false
lock file = /var/lock/object6020.lock

[object6030]
max connections = 25
path = /srv/3/node/
read only = false
lock file = /var/lock/object6030.lock

[object6040]
max connections = 25
path = /srv/4/node/
read only = false
lock file = /var/lock/object6040.lock
" > /etc/rsyncd.conf

sed -i -e "s/disables=syes/disable = no/g" /etc/xinetd.d/rsync

swift settings

認証はtempauthを利用する。

tempauthでは以下の書式でユーザーを管理する。

user_<account>_<user> = <key> [group] [group] [...] [storage_url]

特別なグループとして以下が定義されている。

.reseller_admin:
.admin:

proxy-server.conf

echo "
[DEFAULT]
bind_port = 8080
user = swift
log_facility = LOG_LOCAL1

[pipeline:main]
pipeline = healthcheck cache tempauth proxy-server

[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true
account_autocreate = true

[filter:tempauth]
use = egg:swift#tempauth
user_admin_admin = admin .admin .reseller_admin
user_test_tester = testing .admin
user_test2_tester2 = testing2 .admin
user_test_tester3 = testing3

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:cache]
use = egg:swift#memcache
" > /etc/swift/proxy-server.conf
echo "
[swift-hash]
# random unique string that can never change (DO NOT LOSE)
swift_hash_path_suffix = changeme
" > /etc/swift/swift.conf

for x in {1..4}; do

    echo "[DEFAULT]
devices = /srv/$x/node
mount_check = false
bind_port = 60${x}2
user = swift
log_facility = LOG_LOCAL2

[pipeline:main]
pipeline = account-server

[app:account-server]
use = egg:swift#account

[account-replicator]
vm_test_mode = yes

[account-auditor]

[account-reaper]
" > /etc/swift/account-server/$x.conf

    echo "[DEFAULT]
devices = /srv/$x/node
mount_check = false
bind_port = 60${x}1
user = swift
log_facility = LOG_LOCAL2

[pipeline:main]
pipeline = container-server

[app:container-server]
use = egg:swift#container

[container-replicator]
vm_test_mode = yes

[container-updater]

[container-auditor]

[container-sync]
" > /etc/swift/container-server/$x.conf

    echo "[DEFAULT]
devices = /srv/$x/node
mount_check = false
bind_port = 60${x}0
user = swift
log_facility = LOG_LOCAL2

[pipeline:main]
pipeline = object-server

[app:object-server]
use = egg:swift#object

[object-replicator]
vm_test_mode = yes

[object-updater]

[object-auditor]
" > /etc/swift/object-server/$x.conf

done

client setup

mkdir ~/bin
git clone https://github.com/openstack/swift.git
cd ~/swift
#python setup.py develop
python setup.py install
echo "
export SWIFT_TEST_CONFIG_FILE=/etc/swift/func_test.conf
export PATH=${PATH}:~/bin
" >> ~/.bashrc
. ~/.bashrc

setting up scripts for running Swift

echo "
#!/bin/bash

swift-init all stop
find /var/log/swift -type f -exec rm -f {} ;
sudo umount /mnt/sdb1
sudo mkfs.xfs -f -i size=1024 /dev/sdb1
sudo mount /mnt/sdb1
sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
sudo chown swift:swift /mnt/sdb1/*
sudo -u swift mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4
sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log /var/log/syslog
sudo service rsyslog stop
sudo service rsyslog start
sudo service memcached stop
sudo service memcached start
" > ~/bin/resetswift

echo "
#!/bin/bash

cd /etc/swift

rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz

swift-ring-builder object.builder create 18 3 1
swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1
swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1
swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1
swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1
swift-ring-builder object.builder rebalance
swift-ring-builder container.builder create 18 3 1
swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1
swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1
swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1
swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1
swift-ring-builder container.builder rebalance
swift-ring-builder account.builder create 18 3 1
swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1
swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1
swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1
swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1
swift-ring-builder account.builder rebalance
"  > ~/bin/remakerings

echo "
#!/bin/bash

swift-init main start
" > ~/bin/startmain

echo "
#!/bin/bash

swift-init rest start
" > ~/bin/startrest

chmod +x ~/bin/*

remakerings

#cd ~/swift; ./.unittests

service start

./bin/startmain

test1

curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0
curl -v -H 'X-Auth-Token: <token-from-x-auth-token-above>' <url-from-x-storage-url-above>

test swift client

こっちのがらくらく。

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
echo "hogehoge" > test.txt
swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing upload testcontainer test.txt
swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
rm -f test.txt
swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing download testcontainer test.txt
cat test.txt

Usage: swift <command> [options] [args]

Commands:
  stat [container] [object]
    Displays information for the account, container, or object depending on the
    args given (if any).
  list [options] [container]
    Lists the containers for the account or the objects for a container. -p or
    --prefix is an option that will only list items beginning with that prefix.
    -d or --delimiter is option (for container listings only) that will roll up
    items with the given delimiter (see Cloud Files general documentation for
    what this means).
  upload [options] container file_or_directory [file_or_directory] [...]
    Uploads to the given container the files and directories specified by the
    remaining args. -c or --changed is an option that will only upload files
    that have changed since the last upload. -S <size> or --segment-size <size>
    and --leave-segments are options as well (see --help for more).
  post [options] [container] [object]
    Updates meta information for the account, container, or object depending on
    the args given. If the container is not found, it will be created
    automatically; but this is not true for accounts and objects. Containers
    also allow the -r (or --read-acl) and -w (or --write-acl) options. The -m
    or --meta option is allowed on all and used to define the user meta data
    items to set in the form Name:Value. This option can be repeated. Example:
    post -m Color:Blue -m Size:Large
  download --all OR download container [options] [object] [object] ...
    Downloads everything in the account (with --all), or everything in a
    container, or a list of objects depending on the args given. For a single
    object download, you may use the -o [--output] <filename> option to
    redirect the output to a specific file or if "-" then just redirect to
    stdout.
  delete --all OR delete container [--leave-segments] [object] [object] ...
    Deletes everything in the account (with --all), or everything in a
    container, or a list of objects depending on the args given. Segments of
    manifest objects will be deleted as well, unless you specify the
    --leave-segments option.

Example:
  swift -A https://auth.api.rackspacecloud.com/v1.0 -U user -K key stat

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -s, --snet            Use SERVICENET internal network
  -v, --verbose         Print more info
  -q, --quiet           Suppress status output
  -A AUTH, --auth=AUTH  URL for obtaining an auth token
  -V AUTH_VERSION, --auth-version=AUTH_VERSION
                        Specify a version for authentication(default: 1.0)
  -U USER, --user=USER  User name for obtaining an auth token
  -K KEY, --key=KEY     Key for obtaining an auth token

service enable

とりあえずはONにしない。

chkconfig memcached on
chkconfig openstack-swift-account on
chkconfig openstack-swift-container on
chkconfig openstack-swift-object on
chkconfig openstack-swift-proxy on

https

swift で HTTPS ! | ちからの備忘録的日記

cd /etc/swift
openssl req -new -x509 -nodes -out cert.crt -keyout cert.key -days 365
  • proxy-server.confにcert_fileとkey_fileを追加する。
  • bind_portは8080でもいいけど、443にしてみる。

vi /etc/swift/proxy-server.conf

[DEFAULT]
bind_ip = 0.0.0.0
bind_port = 443
user = swift
log_facility = LOG_LOCAL1
cert_file = /etc/swift/cert.crt
key_file = /etc/swift/cert.key

[pipeline:main]
pipeline = healthcheck cache tempauth proxy-server

[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true
account_autocreate = true

[filter:tempauth]
use = egg:swift#tempauth
#user_<account>_<user> = <key> [group] [group] [...] [storage_url]
user_admin_admin = admin .admin .reseller_admin
user_test_tester = testing .admin
user_vol01_admin = admin .admin https://192.168.200.12:443/v1/AUTH_vol01
user_vol01_user01 = user01 https://192.168.200.12:443/v1/AUTH_vol01
user_vol02_admin = admin .admin
user_vol02_user01 = user01
user_vol02_user02 = user02

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:cache]
use = egg:swift#memcache
swift-init proxy restart

curl でテストするときは -k をつけてSSLの正当性を確認しないようにする。

GUIでswiftにアクセスする。cyberduck

cyberduck で swift ライフ満喫!したかった・・・ | ちからの備忘録的日記

cyberduckで接続できるみたいなので試したところできました。

ただし、Win,Macともに設定ファイルを書き換える必要があります。

help/en/howto/openstack – Cyberduck

swauthの場合と違い、レスポンスに含まれるURLはtempauthの設定の最後のstorage_urlになるっぽい。何も書かないと127.0.0.1で返しているのかも。

macのcyberduck

~/Library/Preferences/ch.sudo.cyberduck.plist に以下を追加

cf.authentication.context = "/auth/v1.0";

そうすると、認証でエラーになっていたのが、一回エラーにはなるけど接続できるようになった。

windowsのcyberduck

各環境ごとにファイルの位置が違うけども、

  • Windows 7. C:Users[username]AppDataRoamingCyberduck
  • Windows Vista. C:Users[username]AppDataRoamingCyberduck
  • Windows XP: C:Documents and Settings[username]Local SettingsApplication DataCyberduck

どれかを開いて以下を追加する。

<setting name="cf.authentication.context" value="/auth/v1.0" />

cyberduck参考

以下のhowtoを見ると、macの場合は

defaults write ch.sudo.cyberduck cf.authentication.context "/auth/v1.0"

と、Terminalから実行すればよさそう。だけど、なんか怖いのでやってない。

bind_ip

他のホストからアクセスしたい場合。

/etc/swift/proxy-server.conf

[DEFAULT]
bind_ip = 0.0.0.0

ただし、

-A INPUT -i eth0 -j REJECT --reject-with icmp-host-prohibited

という設定になってるとswiftコマンドでうまく接続できない。
tcdumpでみると

ICMP host ほすと unreachable - admin prohibited

ということは、なんかしてるの?

操作

一覧を表示

権限の変更

まず前提として以下の設定とする。

tempauthの権限設定

user_vol01_readmin = readmin .admin .reseller_admin https://192.168.200.12:443/v1/AUTH_vol01
user_vol01_admin = admin .admin https://192.168.200.12:443/v1/AUTH_vol01
user_vol01_user01 = user01 .group01 https://192.168.200.12:443/v1/AUTH_vol01
user_vol01_user02 = user01 .group01 https://192.168.200.12:443/v1/AUTH_vol01
user_vol01_user03 = user03 .group02 https://192.168.200.12:443/v1/AUTH_vol01

そして、入力が面倒なのでデフォルトはvol01:adminとしてswiftコマンドを簡単に使う。

export ST_AUTH=https://192.168.200.12/auth/v1.0
export ST_AUTH_VERSION=1.0
export ST_USER=vol01:admin
export ST_KEY=admin

container01を作成する。
* .r:*
* .rlistings
* group01にcontainer01の読み込み権限に設定する。(これよくわかってない)

swift post container01 -r '.r:*, .rlistings' -w 'vol01:user01'

確認

% swift stat container01
  Account: AUTH_vol01
Container: container01
  Objects: 1
    Bytes: 0
 Read ACL: .r:*,.rlistings
Write ACL: vol01:user01
  Sync To:
 Sync Key:
Accept-Ranges: bytes

確認

% swift upload container01 test1
test1
% swift list container01
test1
% swift -A https://192.168.200.12/auth/v1.0 -U vol01:user01 -K user01 list container01
test1
% swift -A https://192.168.200.12/auth/v1.0 -U vol01:user01 -K user01 download container01 test1
  • write-aclについてはいまいちよくわかっていない。この設定ではuser01で書き込みできない。
  • .adminが書ければ十分だけど、設定がいまいちわからない。
  • それよりも.rlistingsを付与しないとlistできないのだが、記述が少なすぎてぴよる。

メモ
* listings