...A Five-Year Study of File-System Metadata Nitin Agrawal University of Wisconsin, Madison nitina@cs.wisc.edu William J. Bolosky, John R. Douceur, Jacob R. Lorch Microsoft Research {bolosky,johndo,lorch}@microsoft.com Abstract For five years, we collected annual snapshots of filesystem metadata from over 60,000 Windows PC file systems in a large corporation. In this paper, we use these snapshots to study temporal changes in file size, file age, file-type frequency, directory size, namespace structure, file-system population, storage capacity and consumption, and degree of file modification. We present a generative model that explains the namespace structure and the distribution of directory sizes. We find significant temporal trends relating to the popularity of certain file types, the origin of file content, the way the namespace is used, and the degree of variation among file systems, as well as more pedestrian changes in sizes and capacities. We give examples of consequent lessons for designers of file systems and related software. characteristics of file systems, including file and directory population, storage capacity, storage consumption, and degree of file modification. The contributions of this work are threefold. First, we contribute the collected data set, which we will sanitize and make available for general use later this year. This is the largest set of file-system metadata ever collected, and it spans the longest time period of any sizeable metadata collection. To obtain this...
Words: 10092 - Pages: 41
...Microsoft Windows File Systems Jon Bartholf POS421: Windows Server Networking December 13, 2010 Ali Shokraee TABLE OF CONTENTS INTRODUCTION..................................................................................................................... 3 FILE AND FILE SYSTEM: WHAT THEY ARE.................................................................. 3 FILE ALLOCATION TABLE (FAT)...................................................................................... 4 FAT12 (12-bit)....................................................................................................................... 4 FAT16 (16-bit)....................................................................................................................... 4 FAT32 (32-bit)....................................................................................................................... 5 HIGH PERFORMANCE FILE SYSTEM (HPFS)................................................................ 5 NEW TECHNOLOGY FILE SYSTEM (NTFS).................................................................... 5 CONCLUSION.......................................................................................................................... 6 APPENDIXES............................................................................................................................ 7 Appendix A: File Allocation Table (FAT) Bit and Usage (Figure...
Words: 1524 - Pages: 7
...Jeff Knight_Lab4A.doc Instructor * Differentiating Between File Systems Which File System Is Intended Primarily For Use in USB Flash Drives FAT32 refers to the 32-bit version of FAT or File Allocation Table. FAT is a computer filing system typically used in a variety of portable devices including floppy disks, memory cards, and USB flash drives among others. This filing system is very simple which is why it is commonly used on portable devices. But this particular filing system can also be applied to older operating systems including that of MS-DOS and Windows 9x versions. FAT32 uses disk space more efficiently than older file allocation standards or formats. It is also capable of handling drives of up to 2 TB in size. And even if drives have bigger sizes, the FAT32 system allows for the use of smaller clusters which results to better use of disk space. The FAT32 file allocation system also allows relocation of the root folder or the (highest folder in the hierarchy). and at the same time utilizes the backup copy of the file allocation table in doing this operation. This increases FAT32’s robustness and efficiency. Having numerous root folders at various locations is also not a concern with the FAT32 system. This results in the possibility of resizing the FAT32 partitions dynamically. But not all systems are compatible with FAT32 including old types of motherboards and BIOS. Some also...
Words: 906 - Pages: 4
...* Microsoft Windows File Systems University of Phoenix POS/421 July 23rd 2012 * * * * * * * * Microsoft Windows File Systems File systems are representations of arranged data and metadata that define the stored data. Microsoft Windows has used three types file systems, File Allocation Table (FAT), FAT 32 and since Windows 2000 and Windows NT 4.0 with Service Pack 4 or later the standard has been New Technology File System (NTFS) (Microsoft, 2012). File systems are also be a part of disk partitions, which are isolated storage segments that can also be used for running multiple operating systems. FAT FAT is the original and most simplistic form of file systems and because of this it has many limitations, but in some cases it may be the only option. According to LLC SysDev Laboratories (2012), FAT, “Consists of file system descriptor sector (boot sector or superblock), file system block allocation table (referenced as File Allocation Table) and plain storage space to store files and folders.” Directories stored in FAT systems use arrays of 32 byte records. The simplest form of FAT is FAT12 which was only used for old floppy disks, next FAT16 also known as just FAT and FAT32 are mostly used for memory cards and USB devices, digital cameras and mobile phones (LLC SysDev Laboratories, 2012). FAT is limited to a maximum of a 4 GB drive and FAT 32 is supported by drives...
Words: 711 - Pages: 3
...Differentiating Between File Systems Chrishara Cobb NT1110 Differentiating Between File Systems Windows has tree file systems used to format USB flash drives, Fat 16, Fat 32 and NTFS. FAT32 is primarily used for large USB flash drives. While NTFS may be faster, and more advanced it may cause the USB to not be able to be used on non-Window’s systems. Over time it may cause the Flash drive to crash and have permission problems. If you have permission issues it can stop you from being able to save work on the flash drive. While these are serious issues that can happen there are advantages to with using NTFS. Files on flash drives are more secure, and allow you to view them anywhere. NTFS also allows encryption access so that if you have a file on your flash drive that you don’t want everyone to be able not read or access if your flash drive is ever lost, you can protect it with a password in order for it to be opened. FAT32 is considered over a NTFS because it does not have some of the issues NTFS has with security. Which basically means that any file that you have on your system can be read by anyone. FAT32 can also be read by almost all OS including open source like; Linux MAC OS. When deciding what File System is best, you have to factor is long term reliability. Will your data be safe and always accessible? Is have the option to encrypt your files worth the headache of only being able to use the Flash drive on Widow’s...
Words: 271 - Pages: 2
...File System Comparison POS/421 Windows Server Networking January 30, 2012 Ali Shokraee File System Comparison This paper is going to compare the different file system structures used for the different version of Windows Operating Systems. This will go from the Disk Operating System (DOS) of the early years to the latest Windows Server 2008. The desktop OS and the servers OS will be compared and contrasted in this paper. Different Types of File Systems There were many different types of file systems that were used over the years that Microsoft was in business. This ranges from the DOS years to the most current version from Microsoft, Windows Server 2008. The following table (table 1) shows the different File Systems that Microsoft has used and the Operating System (OS) that uses that type of File System (NTFS.com, 2011). |File System Structure |Operating System | |FAT12 |All Versions of MS DOS | |FAT16 |All Versions of MS DOS | |FAT32 |DOS v7 and Higher, Windows 98, Windows ME, Windows 2000, Windows XP, Windows 2003 | | |Server, Windows Vista, Windows 7 | ...
Words: 1183 - Pages: 5
...S. Park and S.-Y. Ohm: New Techniques for Real-Time FAT File System in Mobile Multimedia Devices 1 New Techniques for Real-Time FAT File System in Mobile Multimedia Devices Sunhwa Park and Seong-Young Ohm Abstract — Flash memory has become the most important storage media in the mobile multimedia products such as MP3 players, mobile phones, and digital cameras. Most mobile multimedia devices, however, use one of the conventional FAT file systems slightly modified for flash memory environments. Our analysis shows that theses file systems have some restriction in recording a live multimedia stream stably because they have irregular write response times. In this paper, we have considered the problems of the existing FAT file system and propose two new techniques to solve the problems. The first technique called Sector Reservation method reduces internal overhead effectively. And the other method called ACPA avoids the periodic cluster allocation of the conventional FAT file system and removes the frequent modifications on the file allocation table in the FAT file system. To evaluate our new techniques, we implemented a prototype Real-Time FAT file system on ARM9 board with our two novel techniques. The experimental results show that our system achieves our goal successfully in that its write response times are very deterministic and more uniform1. Index Terms — Flash memory, FAT file system, FTL, Mobile multimedia device. I. INTRODUCTION One of the distinct trends in...
Words: 7197 - Pages: 29
...In differentiating between the File Allocation Table (Fat) 32 and New Technology File system (NTFS) file systems it is important to know what each is. In this assignment I will define in more detail what each system is. The file system that is primarily for use in large USB flash drives is the FAT system which uses an 8KB allocation unit size for drives as large as 6GB, and a maximum logical partition size for as large as 16 GB. It can also be used to format hard disks, flash memory, and removable media. The NTFS provides can provide support for anywhere from 2 TB to the maximum of 16 Exabyte’s. It has access control, built in compression, and individual recycle bins. It also supports the encrypting file system file and mounting drive. It has a disk quota that support hot swapping and indexing as well. FAT 32 systems are not necessarily preferred over the NTFS due to the fact that the NTFS has more security features as well as other features that the FAT 32 does not have. However there is one thing that it does have that can make it more preferred than the NTFS, this being the fact that it can be used on the older systems where the NTFS cannot be. That and the fact that it is also used for floppy disk use, the backward compatibility makes it more preferred as well. The use of Windows in multi boot systems is another advantage that it has over the...
Words: 255 - Pages: 2
...the fly without having to boot into a special disk utility. The Disk Management utility can only manage file systems compatible with Windows operating systems, such as FAT16/FAT32 and NTFS. Let’s say you want to make a partition smaller so you can create another partition on a hard drive this utility will allow you to do so. ScanDisk- A DOS utility included only in Windows 95/98/ME it is used to scan the suface ofthe hard disk and repair errors on the hard drive scan disk scans for lost clusters on the hard drive and turns them into .chk files. Chkdisk -is a command line program that also scans the hard drive and looks for lost clusters on the hard disk. It reports the current amount of free memory and disk space. If you use the chkdsk /f command which is Chkdsk with the /f = fix command it reclaims the lost clusters and turns them into .CHK files. FDisk- Is a program that runs in MS-DOS and non-NT versions of Windows and creates partitions on the hard drive Windows 95b first added support for FAT-32 partitions into fdisk. Before that it only supported partitions up to 2 GB using FAT-16. Disk Manager- A driver first introduced in 1985 from Ontrack Data Recovery, a Kroll Ontrack subsidiary, Eden Prairie, MN (www.ontrack.com) that allowed older DOS PCs with a 32MB disk limit to access larger drives. Since PCs made before 1994 may have had a system BIOS that did not support larger drives, the Disk Manager utility grew in popularity and was often bundled with...
Words: 671 - Pages: 3
... INTODUCTION TO UNIX 1.1 Single-User Systems The personal computer (PC) is a small General-purpose system that can execute programs to perform a wide variety of tasks. The PC, however, was designed for use by one person at a time, that is, it is Single-User oriented with MS-DOS as the de facto standard operating system for this range of machines. Single user systems became very popular due to the low cost hardware and wide range of software available for these machines. 1.2 Multi-User Systems As opposed to single-user systems there are also larger systems, which more than one person can use at any time. Such systems are referred to as multi-user systems. Multi-user systems would be required when a number of applications have to be run simultaneously, or common resources, like printers and disks, are to be shared by a number of users. 1.3 Hardware – Multi-User Systems While the hardware components of a multi-user system are similar to that of a singleuser system, the following differences should be noted. The CPU of a multi-user system is more powerful and has capabilities to support multi-programming and multi-tasking, two features essential for multi-user systems. The Hard disk of a multi-user system is bigger in capacity. Most multi-user systems use magnetic tape as external storage for backup of software. Single-user systems use floppies as the backup device. This is because multi-user systems have large hard disks, which have to be backed...
Words: 8312 - Pages: 34
...binaries Holds the files needed to bring the system up and run it when it first comes up in single-user mode. Holds system utilities /boot Static files of the boot loader Contains all of the files needed to boot the system. /dev Device files /etc Machine–local system configuration files One of the most important is /etc/passwd /etc/X11 Machine–local configuration files for the X Window System /home User home directories /lib Shared Libraries /lib/modules Loadable kernel modules /mnt Mount point for temporarily mounting filesystems /opt Add-on software packages. (optional) /proc Kernel and process information virtual filesystem /root home directory for root /sbin Essential system binaries Utilities used for system administration are stored in /sbin and /usr/sbin. /sbin *also /sbin directory includes utilities needed during the booting process /usr/sbin holds utilities used after the system is up and running Older versions of linux In older versions of Linux, many system administration utilities were scattered through several directories that often included other system files (/etc, /usr/bin, /usr/adm, /usr/include). /sys Device pseudofilesystem /tmp temporary files /usr Second major hierarchy. Traditionally includes subdirectories that contain information used by the system. Files in /usr subdirectories...
Words: 640 - Pages: 3
...1. What is the function of the /etc/hosts file? Which services can you use in place of or to supplement the hosts file? This file holds the hostname and IP address of the local system. It also holds a special entry for localhost. This entry supports the loopback service, which allows the local system to talk to itself. The IP address of the loopback serviceis always 127.0.0.1, while 127.0.1.1 names the local system. 2. What does the /etc/resolv.conf file do? What do the nameserver lines in this file do? The /etc/resolv.conf file is the resolver configuration file. It provides access to DNS for Internet address resolution. The nameserver line indicates which systems the local system should query to resolve hostnames into IP addresses and vice versa. 3. What is an inode? What happens to the inode when you move a file within a filesystem? An inode is a data structure, stored on disk, that defines a file’s existence and is identified by an inode number. An inode contains critical information about a file, such as the UID of the owner, where it is physically located on the disk, and how many hard links point to it. In addition, SELinux stores extended information about files in inodes. An inode that describes a directory file maps each of the filenames in the directory to the inode that describes that file. This setup allows an inode to be associated with more than one filename and to be pointed to from more than one directory. 4. What does the .. entry...
Words: 682 - Pages: 3
...1. / – Root * Every single file and directory starts from the root directory. * Only root user has write privilege under this directory. * Please note that /root is root user’s home directory, which is not same as /. 2. /bin – User Binaries * Contains binary executables. * Common linux commands you need to use in single-user modes are located under this directory. * Commands used by all the users of the system are located here. * For example: ps, ls, ping, grep, cp. 3. /sbin – System Binaries * Just like /bin, /sbin also contains binary executables. * But, the linux commands located under this directory are used typically by system aministrator, for system maintenance purpose. * For example: iptables, reboot, fdisk, ifconfig, swapon 4. /etc – Configuration Files * Contains configuration files required by all programs. * This also contains startup and shutdown shell scripts used to start/stop individual programs. * For example: /etc/resolv.conf, /etc/logrotate.conf 5. /dev – Device Files * Contains device files. * These include terminal devices, usb, or any device attached to the system. * For example: /dev/tty1, /dev/usbmon0 6. /proc – Process Information * Contains information about system process. * This is a pseudo filesystem contains information about running process. For example: /proc/{pid} directory contains information about the process with that particular pid. * This is a virtual filesystem...
Words: 578 - Pages: 3
...permission. Taking advantage of Ext3 journaling file system in a forensic investigation AD Copyright SANS Institute Author Retains Full Rights Taking advantage of Ext3 journaling file system in a forensic investigation fu ll r igh ts. Taking advantage of Ext3 journaling file system in a forensic investigation rr eta ins GCFA Gold Certification Author: Gregorio Narváez, gnarvae@yahoo.com Adviser: Paul Wright th 07 ,A ut ho Accepted: December 30 , 2007 © SA NS In sti tu te 20 Key fingerprint = AF19 FA27 2F94 998D FDB5 DE3D F8B5 06E4 A169 4E46 Gregorio Narváez © SANS Institute 2007, 1 As part of the Information Security Reading Room Author retains full rights. Taking advantage of Ext3 journaling file system in a forensic investigation TABLE OF CONTENT 1. INTRODUCTION............................................................................................................................... 3 1.1 Lab Setup .................................................................................................................................... 3 2. EXT3FS JOURNAL FUNDAMENTALS......................................................................................... 4 fu ll r igh ts. 2.1 Journal Life Cycle....................................................................................................................... 6 2.2 File Deletion Process: Ext2 Vs Ext3....................
Words: 15276 - Pages: 62
...edhat® ® Te r r y C o l l i n g s & K u r t W a l l UR ON IT OOLS IN Y T C E CD-R L TH O ED UD M Linux Solutions from the Experts at Red Hat ® ® P R E S S™ SEC Red Hat® Linux® Networking and System Administration Red Hat® Linux® Networking and System Administration Terry Collings and Kurt Wall M&T Books An imprint of Hungry Minds, Inc. Best-Selling Books G Digital Downloads G e-Books G Answer Networks e-Newsletters G Branded Web Sites G e-Learning New York, NY G Cleveland, OH G Indianapolis, IN Red Hat® Linux® Networking and System Administration Published by Hungry Minds, Inc. 909 Third Avenue New York, NY 10022 www.hungryminds.com Copyright © 2002 Hungry Minds, Inc. All rights reserved. No part of this book, including interior design, cover design, and icons, may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording, or otherwise) without the prior written permission of the publisher. Library of Congress Control Number: 2001093591 ISBN: 0-7645-3632-X Printed in the United States of America 10 9 8 7 6 5 4 3 2 1 1O/RT/QT/QS/IN Distributed in the United States by Hungry Minds, Inc. Distributed by CDG Books Canada Inc. for Canada; by Transworld Publishers Limited in the United Kingdom; by IDG Norge Books for Norway; by IDG Sweden Books for Sweden; by IDG Books Australia Publishing Corporation Pty. Ltd. for Australia and New Zealand; by TransQuest Publishers Pte Ltd. for Singapore, Malaysia, Thailand...
Words: 220815 - Pages: 884