Content-type: text/html Manpage of plfsrc

plfsrc

Section: File Formats (5)
Updated: 12/3/2010
Index Return to Main Contents

NAME

plfsrc - the configuration file for PLFS (Parallel Log Structured File System)

DESCRIPTION

This configuration file defines the mapping between logical and physical paths on the parallel filesystem. It also has some configuration options that PLFS will use to try and optimize performance. This file is never written by PLFS, it is only read and it is the duty of the systems administrator or user to properly create and maintain this file. Please note that this configuration file is very important; changing it can potentially disrupt the ability to read existing files. If a plfsrc file must be changed, it is recommended to create a new mount point and copy files over. Additionally, please use the 'plfs_check_config' tool included in the installation after every edit to the plfsrc file. PLFS searches for this file in $ENV{'PLFSRC'}, $HOME/.plfsrc, /etc/plfsrc in that order.

FILE FORMAT

The file consists of one keyword argument pair per line. Configuration options must be separated by whitespace, ie "keyword argument". Any line with starting with a # is not interpreted. Path names must be fully qualified. Note that all keywords should be in lower-case. The possible keywords and their meanings are as follows:

include

: This allows multiple plfsrc files to be parsed. This is useful if you want to define global values and then put individual mount points in individual plfsrc files. You might do so if some mount points are shared on different clusters (e.g. at LANL we have compute nodes and file transfer nodes and we want different global values for things like threadpool_size but we want to ensure that their mount points are the same so we have individual plfsrc files for each but then include the same plfsrc files that define shared mount points.

num_hostdirs

: This should be the square root of the total number of compute nodes typically expected to participate in a job. Optional. Default is 32. Max is 1024.

threadpool_size

: This value is set to the number of threads to run on a machine; for machines used in large jobs, this value should probably be around 2; for machines used in small jobs (like file transfer nodes), this value should probably be around 16. Optional. Default is 8.

index_buffer_mbs

: This is the amount of memory (in megabytes) that PLFS can use to buffer indexing information while it is writing files. This buffer, if sufficiently large, enables subsequent optimizations which improve read open times. This behavior is only available through the MPI-IO interface and can be controlled by an optional hint of "plfs_flatten_close." When setting this value, bear in mind that every open file handle can buffer this much memory so that total memory being used can be much larger than this value. Optional. Default is 64.

lazy_stat

: The lazy_stat flag, when it is set as 0 plfs will do slow stat(getattr). Default is 1.

mount_point

: The path which users use to access PLFS files. Users should NEVER see the backends. Note this option must appear in the file before the backends keyword. Required.

backends

: The location where you want the physical containers to reside for each mount_point. You can have a comma-separated list which will distribute PLFS workload across multiple backend directories. LANL uses this feature to spread files across multiple volumes and thereby balance load across different metadata servers. Note this option must appear after the mount_point keyword. Required.

workload

: This tells PLFS whether a specific mount_point is intended for either shared file workloads (i.e. N-1) or file-per-proc workloads (i.e. N-N). Possible values are either "shared_file" or "file_per_proc". "n-1" can also be used in place of "shared_file" and "n-n" can also be used in place of "file_per_proc. shared_file is intended for workloads where multiple writers write concurrently to a single file. Other workloads should use "file_per_proc" which should make a large metadata improvement for most metadata workloads so long as multiple backends are defined for the mount point. This is especially true if each backend is serviced by a different metadata server so that the metadata workload can be balanced across multiple servers.

canonical_backends

: This is an advanced directive and will not be used in typical installations. See shadow_backends for a more detailed description.

shadow_backends

: This is an advanced directive and will not be used in typical installations. This, along with canonical_backends, allows the user to specify that some set of backends are where canonical containers should be placed and some different set of backends are where shadows should be placed. If these directives are used, then shadow subdirs will ALWAYS be created in shadow backends. This can be used in instances where the canonical is desired to be on slow global and the data pieces are desired to be on fast local. In scenarios with lots of different fast local mounts, different nodes will have different sets of shadow_backends defined so that each node always writes to the fastest possible backend. For reading, this will require however that all locations are visible everywhere.

syncer_ip

: This is an advanced directive and will not be used in typical installations. This is used in conjunction with shadow_backends and canonical_backends. It is possible to use an RPC server to asynchronously copy data from shadow_backends into canonical_backends by specifying this value and by calling plfs_protect.

global_summary_dir

: The path into which to drop summary information. Useful to determine how much PLFS is being used and a bit of information about how it is being used. This must be set to a globally writable and globally visible path. When this is set, each close() will drop a file into this directory with some statistical information in the filename. This will happen on every close() for every proc when using FUSE and just on rank 0's close() when using ADIO. Be careful if you use this because this directory can get very full and can cause performance problems. Optional.

statfs

: The path on which to resolve statfs calls received through FUSE. Typically this is not specified but LANL uses this to avoid hanging on mounts when backend volumes are not specified. This is because FUSE calls statfs when it mounts and PLFS by default forwards that to one of the backends and this can then hang. But the LANL admins don't want anything in the boot cycle to hang so they can use this command to force PLFS to forward statfs to a path other than the backends which won't hang. Optional.

test_metalink

: This is only for developers to do testing. It won't be explained here. If you really want to know what it does, please read the code. Otherwise, do not use this directive.
Note the mount_point and backend may be specified multiple times in the same file. Note that some commands are global and some are unique to a specific mount point. This is to allow different backend declarations for each mount point for example. Indeed it is an error to share a backend in multiple backends.

mlog_defmask

: The default logging level. This is used to control how much logging is enabled. Possible values: EMERG, ALERT, CRIT, ERR, WARN, NOTE, INFO and DEBUG. The default is WARN.

mlog_setmasks

: Resets the logging level for a subsystem (overwriting the default value). Currently defined subsystems are: plfs_misc, internal, container, index, writefile, fileops, utilities, store, FUSE, and MPI. If the subsystem is omitted, then it sets the log mask for all subsystems. For example, "INFO,index=DEBUG,container=DEBUG" sets the mask for all subsystems to INFO, and then sets index and container to DEBUG.

mlog_stderrmask

: Level at which log messages are printed to stderr. Useful if set to a higher priority than the normal mask. This allows users to only have a subset of higher priority log messages printed to stderr. The default value is CRIT.

mlog_stderr

: If set to 1, causes all logged messages to be printed to stderr, irrespective of the value of mlog_stderrmask (e.g. for debugging). The default value is 0.

mlog_file

: If set, then this contains the name of the PLFS log file. If not set (the default), then logs are not saved to a file. Macros of %p, %h, and %t can be put in the filename; they are expanded to PID, HOSTNAME, and UNIX EPOCH respectively.

mlog_msgbuf_size

: The size, in bytes of the in-memory log message buffer. This is a circular buffer containing the most recent logs. The default value is 4096.

mlog_syslog

: If set to 1, causes all logged messages to be sent the system's syslog program with syslog(3). The default is 0.

mlog_syslogfac

: If mlog_syslog is 1, then this contains the facility level used to open syslog with openlog(3). It must be of the format LOCALn where "n" is between 0 and 7. The default is to use the USER facility.

mlog_ucon

: A low-level debugging option that enables log messages to be sent to a UDP socket (if set to 1). The default is 0.

Note that the mount_point and backend may be specified multiple times in the same file.

A tool for checking the plfsrc file for correct syntax is included in the plfs distribution. See plfs_check_config(1) for more details.

EXAMPLE

A configuration file might appear as follows: num_hostdirs 32
threadpool_size 16
mount_point /tmp/plfs
backends /tmp/.plfs_store

FILES

/etc/.plfsrc

: The system wide configuration file.

~/.plfsrc

: Per user configuration file.

The per user file options will override options in the system wide file. Options in files cannot be combined, only one file will be used.

AUTHORS

PLFS is developed at Los Alamos National Laboratory and is available under LANL LA-CC-08-104. It is hosted at http://sourceforge.net/projects/plfs and more information is available at http://institute.lanl.gov/plfs

This document was created by man2html, using the manual pages.
Time: 14:34:00 GMT, June 15, 2012