分布式文件系统(Distributed File System)是指文件系统管理的物理存储资源不一定直接连接在本地节点上,而是通过计算机网络与节点相连。分布式文件系统的设计基于客户机/服务器模式。一个典型的网络可能包括多个供多用户访问的服务器。另外,对等特性允许一些系统扮演客户机和服务器的双重角色。
FastDFS是一个开源的轻量级分布式文件系统,它对文件进行管理,功能包括:文件存储、文件同步、文件访问(文件上传、文件下载)等,解决了大容量存储和负载均衡的问题。特别适合以文件为载体的在线服务,如相册网站、视频网站等等。
一:安装FastDFS
1、首先安装libevent。
wget http://monkey.org/~provos/libevent-2.0.11-stable.tar.gz
tar -zxvf libevent-2.0.11-stable.tar.gz
cd libevent-2.0.11-stable
./configure --prefix=/usr
make clean
make
make install
2、安装FastDFS的步骤
wget http://fastdfs.googlecode.com/files/FastDFS_v3.03.tar.gz
tar -zxvf FastDFS_v3.03.tar.gz
cd FastDFS
vim make.sh 将#WITH_HTTPD=1 修改成WITH_HTTPD=1 以支持 http
./make.sh
./make.sh install
二:FastDFS的配置
1、配置及启动Tracker Server
mkdir /home/fastdfs
vi /etc/fdfs/tracker.conf
base_path=/home/yuqing/fastdfs -> base_path=/home/fastdfs
http.server_port=8080 -> http.server_port=9090
##include http.conf -> #include http.conf
reserved_storage_space = 4GB -> reserved_storage_space = 1GB
cd /usr/local/bin
fdfs_trackerd /etc/fdfs/tracker.conf
检查tracker是否启动成功,可以查看如下文件/home/fastdfs/logs/trackerd.log
[2011-10-21 14:29:44] INFO - FastDFS v3.03, base_path=/home/fastdfs, run_by_group=, run_by_user=, connect_timeout=30s, network_timeout=60s, port=22122, bind_addr=, max_connections=256, work_threads=4, store_lookup=2, store_group=, store_server=0, store_path=0, reserved_storage_space=1024MB, download_server=0, allow_ip_count=-1, sync_log_buff_interval=10s, check_active_interval=120s, thread_stack_size=64 KB, storage_ip_changed_auto_adjust=1, storage_sync_file_max_delay=86400s, storage_sync_file_max_time=300s, use_trunk_file=0, slot_min_size=256, slot_max_size=16 MB, trunk_file_size=64 MB
[2011-10-21 14:29:44] INFO - HTTP supported: server_port=9090, default_content_type=application/octet-stream, anti_steal_token=0, token_ttl=0s, anti_steal_secret_key length=0, token_check_fail content_type=, token_check_fail buff length=0, check_active_interval=30, check_active_type=tcp, check_active_uri=/status.html
2、配置及启动Storage Server
mkdir /home/fastdfs/fdfs_storage
cd /etc/fdfs
vi storage.conf
base_path=/home/yuqing/fastdfs -> /home/fastdfs/fdfs_storage
store_path0=/home/yuqing/fastdfs -> store_path0=/home/fastdfs/fdfs_storage
group_name=group1
tracker_server=192.168.209.121:22122 -> tracker_server=173.252.200.228:22122
##include http.conf ->#include http.conf
接下来会出现很多mkdir data path,这是系统在创建数据目录。
[2011-10-21 14:49:20] INFO - FastDFS v3.03, base_path=/home/fastdfs/fdfs_storage, store_path_count=1, subdir_count_per_path=256, group_name=group1, run_by_group=, run_by_user=, connect_timeout=30s, network_timeout=60s, port=23000, bind_addr=, client_bind=1, max_connections=256, work_threads=4, disk_rw_separated=1, disk_reader_threads=1, disk_writer_threads=1, disk_rw_direct=0, buff_size=256KB, heart_beat_interval=30s, stat_report_interval=60s, tracker_server_count=1, sync_wait_msec=50ms, sync_interval=0ms, sync_start_time=00:00, sync_end_time=23:59, write_mark_file_freq=500, allow_ip_count=-1, file_distribute_path_mode=0, file_distribute_rotate_count=100, fsync_after_written_bytes=0, sync_log_buff_interval=10s, sync_binlog_buff_interval=10s, sync_stat_file_interval=300s, thread_stack_size=512 KB, upload_priority=10, if_alias_prefix=, check_file_duplicate=0, FDHT group count=0, FDHT server count=0, FDHT key_namespace=, FDHT keep_alive=0, HTTP server port=8888, domain name=
[2011-10-21 14:49:20] INFO - HTTP supported: server_port=8888, http_trunk_size=262144, default_content_type=application/octet-stream, anti_steal_token=0, token_ttl=0s, anti_steal_secret_key length=0, token_check_fail content_type=, token_check_fail buff length=0
[2011-10-21 14:49:42] INFO - file: storage_param_getter.c, line: 97, storage_ip_changed_auto_adjust=1, store_path=0, reserved_storage_space=1024 MB, use_trunk_file=0, slot_min_size=0, slot_max_size=16 MB, trunk_file_size=64 MB
[2011-10-21 14:49:44] INFO - file: tracker_client_thread.c, line: 304, successfully connect to tracker server 173.252.200.228:22122, as a tracker client, my ip is 173.252.200.228
[2011-10-21 14:50:14] INFO - file: tracker_client_thread.c, line: 1100, tracker server 173.252.200.228:22122, set tracker leader: 173.252.200.228:22122
三:测试及使用FastDFS
1、FastDFS之配置client
vi /etc/fdfs/client.conf
base_path=/home/yuqing/fastdfs-> base_path=/home/fastdfs
tracker_server=192.168.209.121:22122 -> tracker_server=173.252.200.228:22122
http.tracker_server_port=8080 ->http.tracker_server_port=9090
##include http.conf ->#include http.conf
2、测试上传文件
cd ~
fdfs_test /etc/fdfs/client.conf upload FastDFS_v3.03.tar.gz
执行成功后会出现如下提示:
This is FastDFS client test program v3.03
Copyright (C) 2008, Happy Fish / YuQing
FastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/
for more detail.
[2011-10-21 15:14:59] INFO - base_path=/home/fastdfs, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0
tracker_query_storage_store_list_without_group:
server 1. group_name=group1, ip_addr=173.252.200.228, port=23000
group_name=group1, ip_addr=173.252.200.228, port=23000
storage_upload_by_filename
group_name=group1, remote_filename=M00/00/00/rfzI5E6hG_O5wNKxAAVAc72y1Jc.tar.gz
source ip address: 173.252.200.228
file timestamp=2011-10-21 15:14:59
file size=344179
file crc32=3182613655
file url: http://173.252.200.228:9090/group1/M00/00/00/rfzI5E6hG_O5wNKxAAVAc72y1Jc.tar.gz
storage_upload_slave_by_filename
group_name=group1, remote_filename=M00/00/00/rfzI5E6hG_O5wNKxAAVAc72y1Jc_big.tar.gz
source ip address: 173.252.200.228
file timestamp=2011-10-21 15:14:59
file size=344179
file crc32=3182613655
file url: http://173.252.200.228:9090/group1/M00/00/00/rfzI5E6hG_O5wNKxAAVAc72y1Jc_big.tar.gz
3、测试下载文件
在浏览器中,输入上图中的url地址, tracker server会自动重定向到存储文件的storage server,文件下载成功。至此,已经成功搭建了FastDFS,编写你自己的client来进行访问吧:
(责任编辑:IT) |