| Lustre 1.6 Operations Manual |     |
Lustre 1.6 Operations Manual
820-3681-10
|
Contents |
1.1 Introducing the Lustre File System
1.4 Files in the Lustre File System
1.4.1 Lustre File System and Striping
1.7 Lustre Failover and Rolling Upgrades
1.8 Additional Lustre Features
2. Understanding Lustre Networking
2.3 Designing Your Lustre Network
2.3.1 Identify All Lustre Networks
2.3.2 Identify Nodes to Route Between Networks
2.3.3 Identify Network Interfaces to Include/Exclude from LNET
2.3.4 Determine Cluster-wide Module Configuration
2.3.5 Determine Appropriate Mount Parameters for Clients
2.4.2 Module Parameters - Routing
2.5 Starting and Stopping LNET
3.1 Preparing to Install Lustre
3.1.1 Supported Operating System, Platform and Interconnect
3.1.2 Required Tools and Utilities
3.1.3 High-Availability Software
3.1.5 Environmental Requirements
3.2 Installing Lustre from RPMs
3.3 Installing Lustre from Source Code
3.3.2 Create and Install the Lustre Packages
3.3.3 Installing Lustre with a Third-Party Network Stack
4.2 Basic Lustre Administration
4.2.4 Working with Inactive OSTs
4.2.5 Finding Nodes in the Lustre Filesystem
4.2.6 Start a Server Without Lustre Service
4.2.7 Specifying Failout Mode for an OST
4.2.8 Running Multiple Lustre Filesystems
4.2.9 Running the Writeconf Command
4.3 More Complex Configurations
4.4.3 Re-addressing a Failover Node
4.4.4 Local Mounts through Network
4.4.5 Start Lustre on a Client or Server Node, Ignoring the Management Network
4.4.6 Mounting a Lustre Server but No Service
4.4.7 Adding an OSS/MDS to an Existing Filesystem
4.4.8 Security Policies on Some Interface for the MDS
4.4.9 Configure a Client with a Persistent Write-Back Cache
4.4.10 Configure a Replicating Proxy Cluster
4.4.13 Echo Client and Single OST Server
4.4.14 Striped OST Echo Server
4.4.15 Long Term Failure and Removal of OSS Targets
4.5 Configuration State Management
4.6 Lustre Configuration Utilities
5.1 Introduction to Service Tags
5.2.2 Discovering and Registering Lustre Components
5.2.3 Information Registered with Sun
6. Configuring Lustre - Examples
6.1.1 Lustre with Combined MGS/MDT
6.1.2 Lustre with Separate MGS and MDT
7. More Complicated Configurations
7.3 Load Balancing with InfiniBand
7.4 Multi-Rail Configurations with LNET
8.1.1 The Power Management Software
8.1.4 Connection Handling During Failover
8.1.5 Roles of Nodes in a Failover
8.4 Configuring MDS and OSTs for Failover
8.4.1 Configuring Lustre for Failover
8.4.2 Starting/Stopping a Resource
8.4.3 Active/Active Failover Configuration
8.4.4 Hardware Requirements for Failover
8.5 Setting Up Failover with Heartbeat V1
8.7 Setting Up Failover with Heartbeat V2
8.7.2 Configuring the Hardware
8.8 Considerations with Failover Software and Solutions
9.1.2 Creating Quota Files and Quota Administration
9.1.5 Known Issues with Quotas
10.1 Considerations for Backend Storage
10.1.1 Selecting Storage for the MDS and OSS
10.1.2 Reliability Best Practices
10.1.3 Understanding Double Failures with Hardware and Software RAID5
10.2 Insights into Disk Performance Measurement
10.3 Lustre Software RAID Support
11.2 Lustre Setup with Kerberos
11.2.1 Configuring Kerberos for Lustre
11.2.2 Types of Lustre-Kerberos Flavors
12.3 Using Lustre with Multiple NICs versus Bonding NICs
12.4 Bonding Module Parameters
12.6 Configuring Lustre with Bonding
13.2 Upgrading from Lustre 1.4.12 to Latest 1.6.x Version
13.2.1 Prerequisites to Upgrading Lustre
13.2.2 Supported Upgrade Paths
13.2.4 Upgrading a Single File system
13.2.5 Upgrading Multiple File Systems with a Shared MGS
13.3 Upgrading Lustre 1.6.x to the Next Minor Version
13.4 Downgrading from Latest 1.6.x Version to Lustre 1.4.12
13.4.2 Downgrading a File System
14.1 Installing the Lustre SNMP Module
14.2 Building the Lustre SNMP Module
14.3 Using the Lustre SNMP Module
15.1.1 Filesystem-level Backups
15.1.3 Performing File-level Backups
15.2 Restoring from a File-level Backup
15.3 LVM Snapshots on Lustre Target Disks
15.3.1 Creating LVM-based Lustre Filesystem As a Backup
15.3.2 Backing Up New Files to the Backup Filesystem
15.3.3 Creating LVM Snapshot Volumes
15.3.4 Restoring From Old Snapshot
16.2 Running POSIX Tests Against Lustre
16.3 Isolating and Debugging Failures
18.1 Lustre I/O Kit Description and Prerequisites
18.1.2 Prerequisites to Using an I/O Kit
18.4.1 Basic Concepts of LNET Self-Test
18.4.2 LNET Self-Test Concepts
18.4.3 LNET Self-Test Commands
19.2.2 MDS Failure (and Failover)
Part III Lustre Tuning, Monitoring and Troubleshooting
20.3 Options to Format MDT and OST Filesystems
20.3.3 Overriding Default Formatting Options
20.5.1 Setting Readahead and MF
20.5.3 Setting Write-Back Cache
20.6 Large-Scale Tuning for Cray XT and Equivalents
21. Lustre Monitoring and Troubleshooting
21.4 Common Lustre Problems and Performance Tips
21.4.1 Recovering from an Unavailable OST
21.4.2 Write Performance Better Than Read Performance
21.4.3 OST Object is Missing or Damaged
21.4.5 Identifying a Missing OST
21.4.10 Reclaiming Reserved Disk Space
21.4.11 Considerations in Connecting a SAN with Lustre
21.4.12 Handling/Debugging "Bind: Address already in use" Error
21.4.13 Replacing An Existing OST or MDS
21.4.14 Handling/Debugging Error "- 28"
21.4.15 Triggering Watchdog for PID NNN
21.4.16 Handling Timeouts on Initial Lustre Setup
21.4.17 Handling/Debugging "LustreError: xxx went back in time"
21.4.18 Lustre Error: "Slow Start_Page_Write"
21.4.19 Drawbacks in Doing Multi-client O_APPEND Writes
21.4.20 Slowdown Occurs During Lustre Startup
21.4.21 Log Message ‘Out of Memory’ on OST
21.4.22 Number of OSTs Needed for Sustained Throughput
21.4.23 Setting SCSI I/O Sizes
22.1.3 Adaptive Timeouts in Lustre
22.1.5 Free Space Distribution
22.2.1 Client I/O RPC Stream Tunables
22.2.2 Watching the Client RPC Stream
22.2.3 Client Read-Write Offset Survey
22.2.4 Client Read-Write Extents Survey
22.2.5 Watching the OST Block I/O Stream
22.2.6 Using File Readahead and Directory Statahead
22.3.1 RPC Information for Other OBD Devices
23.1.1 Format of Lustre Debug Messages
23.2 Tools for Lustre Debugging
23.2.1 Debug Daemon Option to lctl
23.2.2 Controlling the Kernel Debug Log
23.2.5 Printing to /var/log/messages
23.2.8 Adding Debugging to the Lustre Source Code
23.3 Troubleshooting with strace
23.4.1 Determine the Lustre UUID of an OST
24.1 Querying Filesystem Space
25.1.2 Disadvantages of Striping
25.2 Displaying Files and Directories with lfs getstripe
25.3 lfs setstripe - Setting File Layouts
25.3.1 Changing Striping for a Subdirectory
25.3.2 Using a Specific Striping Pattern/File Layout for a Single File
25.3.3 Creating a File on a Specific OST
25.4.3 Adjusting the Weighting Between Free Space and Location
25.5.1 Making Filesystem Objects Immutable
25.6.1 End-to-End Client Checksums
26.2.1 Configuring Root Squash
26.2.2 Enabling and Tuning Root Squash
26.2.3 Tips on Using Root Squash
27.1 Adding an OST to a Lustre Filesystem
27.2 A Simple Data Migration Script
27.3 Adding Multiple SCSI LUNs on Single HBA
27.4 Failures Running a Client and OST on the Same Machine
27.5 Improving Lustre Metadata Performance While Using Large Directories
29. Lustre Programming Interfaces (man2)
30. Setting Lustre Properties (man3)
31. Configuration Files and Module Parameters (man5)
31.2.2 SOCKLND Kernel TCP/IP LND
31.2.8 Portals LND (Catamount)
32. System Configuration Utilities (man8)
32.5 New Utilities in Lustre 1.6
32.5.3 Utilities to Manage Large Clusters
32.5.4 Application Profiling Utilities
32.5.5 More /proc Statistics for Application Profiling
32.5.6 Testing / Debugging Utilities
32.5.14 ll_recover_lost_found_objs
33.4 Maximum Number of OSTs and MDTs
33.5 Maximum Number of Clients
33.6 Maximum Size of a Filesystem
33.8 Maximum Number of Files or Subdirectories in a Single Directory
33.10 Maximum Length of a Filename and Pathname
33.11 Maximum Number of Open Files for Lustre Filesystems
33.12 OSS RAM Size for a Single OST
| Lustre 1.6 Operations Manual | 820-3681-10 | |
Copyright © 2009 Sun Microsystems, Inc. All rights reserved.