Tag: file management

What is my computer’s file-storage system about?

Lenovo Yoga 3 Pro convertible notebook at Rydges Hotel Melbourne

How is your data organised on your computer, whether on its main disk or any removable storage connected to it>

A computer always needs to be able to hold programs and data in a non-volatile manner so users can get back to this data when they switch the computer on again. Here this has evolved through different methods and technologies that answered these needs in different ways.

What were these technologies that were available for home computers?

Initially, home-computer users used to have to use audio cassette tapes to store this data. Subsequently, the magnetic diskette, commonly known as the floppy disk due to it being like a piece of card, became the preferred storage method for computers. Typically, the better computer setups would end up with two floppy-disk drives so that two disks can be accessed at once.

USB external hard disk

A USB external hard disk

The early 1980s saw some manufacturers offer high-capacity fixed-disk drives, which were known as “hard disks” as a storage option for computers with this being preferred by business users. These storage devices earned this name as them being seen as an alternative to the old floppy disks.

Subsequently, Sony brought forward the hard-shelled 3.5” “micro-floppy” and this was brought out alongside a similar technology offered by Hitachi and a few other companies. It was to provide a higher-capacity smaller data-storage magnetic disk that was more rugged than the previous designs and appealed to the design of highly-portable computers.

The optical disk, which is based on CD technology, came in to being as an affordable software-distribution and large-data-distribution technology during the mid 1990s. Subsequently, solid-state non-moving flash storage came to fruition from the late 90s as a removable storage medium for digital cameras and PDAs but became more viable for regular computers since the late 2000s.

Since the magnetic disk came on the scene, there was an increased importance placed on organising where the data existed on these storage systems, with an emphasis on such concepts as file systems, volumes and folders or directories. This was because the various magnetic-disk systems were becoming more and more capacious and users needed to know where their data existed. Here, the file system effectively became a hierarchical database for the information you store on your computer and provided a logical relationship between the files and where the bits and bytes that represented them existed on the storage medium.

Desktop computers required the ability for the user to insert and remove any removable media at a moment’s notice but this required the user to be sure that all the data that was written to the medium before they could remove it. This is in contrast to what was required of mainframe and similar computer systems where an operator had to type commands to add the disk to the computer’s file system or remove it from the file system as part of physically attaching and detaching these disks.

This concept changed when Apple brought in the Macintosh computer which used the Sony 3.5” microfloppy disks. Here, they allowed you you to insert removable media in to that computer but required you to “drag it to the Trashcan” before the disk could be removed. Some advanced removable disk types like the Zip disk implemented this kind of removal in the Windows and other operating system by providing what has been described as a “VCR-style” eject routine due to its relationship to how you used an audio or video recorder. Here, you pressed the eject button on the disk drive which would cause all the data to be written back to the disk before the disk came out.

Now the modern computer has at least one hard disk and / or solid-state disk fixed inside it along with USB ports being used for connecting USB-connected hard disks or memory keys. You may also be inserting your camera’s SD card in to an SD-card slot on your laptop computer or in to an SD-card reader module that plugs in to your computer’s USB port if you were downloading digital images and videos. Some of you may even have an optical drive integrated in your computer or connected to it via a USB cable and use this for archiving data or playing CDs and DVDs.

Your operating system’s file manager

Windows 10 File Manager - logical volumes

All the logical volumes available to a computer – representing hard disks with their logical partitions along with removable media

The operating system that runs your computer will have a file manager that allows you to discover and load your files or move, copy, rename and delete files amongst the logical volumes available to your computer. In Windows, this used to be known as File Manager, then became known as Windows Explorer but is now known as File Explorer. The Apple Macintosh describes this file manager simply as Finder.

This used to be a command-line task but since the arrival of the Apple Macintosh, the file manager is represented using a graphical user interface which shows a list of files, folders or logical volumes that you are dealing with.

Clicking on a folder or logical volume will bring up a screen to show you what is in that folder or logical volume. But clicking on a file will cause it to be opened by the default application or, in the case of a program, cause that program to run.

Moving or copying files nowadays is simply a drag-and-drop affair where you drag the files from the source to the destination, but you may have to hold down the Shift key or use the right-hand mouse button to modify a default move or copy action.

As well, the modern file managers have a “two-stage” delete action for files on a hard disk or other fixed storage where they end up in a “holding-bay” folder known as the Trashcan or Recycle Bin when you delete them. This is to allow you to find files that you may have unintentionally deleted. But to fully delete them for good, you have to delete the contents of this “holding-bay” folder, something you can do by right-clicking or Ctrl-clicking on this folder to bring up a context menu and selecting an “Empty” option.

What is my computer’s file storage system about

The logical volume

Most operating systems represent as their storage system every logical volume be it a removable disk or each partition of a hard disk as its own element. It was the only way to work in the early days of computing because each fixed or removable disk didn’t hold much in the way of data and was its own element. As hard disks became more capacious, there was a requirement to partition them or break a single physical hard disk in to multiple logical volumes because the operating systems of the early days couldn’t hold much data per volume. You can also set up some operating systems to present a folder on a NAS or file server available to you over a network to appear as a logical volume, a practice that was important before networks were commonplace and personal-computer operating systems could address network resources directly. All removable media are still represented with one logical volume per disk, card or stick.

Each logical volume would have the ability to be given a volume name and be represented as a distinct icon which is part of a “Devices”, “Volumes” or similar cluster in the file-management system that is part of the operating system. The icon is typically a crude representation of the storage medium that the logical volume exists on.

Windows, harking back to the Microsoft MS-DOS days, would also assign each logical volume a “drive letter” owing to the fact that each disk drive on the original IBM PC was assigned its own letter with A and B reserved for the floppy disk drives.

The Apple Macintosh represented on the right side of the Desktop screen a “disk” icon for each logical volume currently available to the system. But recent iterations of the Macintosh’s operating system provided a setting so that all of the logical volumes that represented the computer’s fixed storage didn’t appear as desktop icons.

The mid 1980s showed up a situation where an operating system had to identify what kind of disk a logical volume was on because hard disks were becoming more viable and a computer could have multiple disks of different kinds. This was also being augmented by the arrival of networks and file servers where you could “pool” your files on a common computer with larger storage, and CD-ROMs in the early 90s being a cheap way to deliver large amounts of software and data. Thanks to the graphical user interface, this was represented via an icon that represented the kind of disk being handled.

How are they represented?

In Windows, each logical volume, whether fixed or removable,is represented in Windows Explorer or File Explorer by an icon in the left hand panel under “This PC” or “Computer” or something similar depending on the version. If you click on this icon, you will see a list of all the logical volumes available to your computer.

On the Macintosh, you would normally have each of these volumes represented by an icon on the right hand side of your desktop, where you would click on that volume to invoke a Finder window to see all of the files in this volume. On the other hand, Finder would represent all of the volumes in a separate left-hand-side pane.

In both cases, each logical volume would be represented at least with its logical volume name and icon. With some systems, if there is a device that can hold removable media like an SD card reader, floppy disk drive or an optical drive, you will see that device listed but greyed out or de-emphasised if there is nothing in it.

Some operating systems like MacOS X may represent a removable volume like an SD card, USB memory key or optical disk with a distinct icon to highlight their removeability. This will typically be an “eject” symbol which you can click to safely remove that volume. Windows even lists the “eject” word in the right-click option menu for all of the volumes that are removable.

Folders

The folders that exist on a system disk

All the folders that exist on a hard disk, this time the system disk

The Macintosh and, subsequently, MS-DOS and Amiga brought around the concept of directories or folders as a way of organising data across increasingly-larger data volumes. Here, you could organise the data in to smaller clusters that relate to a common theme or purpose with the ability to create a folder within another folder.

Some operating systems like some versions of the Macintosh operating system allowed you to represent a folder with a graphical icon but this was used mainly by software developers when you installed software on the computer.

But all of the computers typically allocate a special folder on the main logical volume for storing all the programs that you run and, in some cases, even create a temporary folder for keeping data that a program stores on an as-needed basis.

How are they represented

On the graphical-user interface, these were represented as a folder icon that is  a part of how the contents of a logical volume was represented. Clicking on this folder icon will allow you to see the contents of that folder.

What is the main or system disk of your computer?

The Main Disk or System Disk for a Windows computer

The Main Disk or System Disk for a Windows computer

The main disk on your computer, which is a hard disk or fixed solid-state-device, stores all the files that are to do with its operating system and all the applications you run on your computer. Such a disk is listed as C: in Windows or MACINTOSH HD on the Macintosh. It is also described as the system disk or the boot disk because it has the operating system that the computer has to load every time it is started, a process described as the “boot” process.

Where the programs that you run exist

It will also contain the data you create but all of the files needed to run the operating system and the applications will be kept in particular folders. For example, the  “Applications” or “Program Files” is kept aside for the applications and games the user installs, with each application you install creating its own subfolder of that folder. This is while a separate folder like “Windows” or “System” is kept for the operating system’s files. Some operating systems like MacOS may also use another folder for keeping plug-ins, fonts and similar common application resources while others may keep these with the applications / programs folder, usually as a subordinate folder.

Where the Desktop is represented

As well, all the icons and files that you store on the Desktop will be kept on a “Desktop” folder which represents everything that exists there.

The data you have created

But you will also end up with user-data space like “Documents”, “Photos” and the like where you save all of the data you create with your computer’s applications. Your e-mail program may store your emails in that folder or in a separate folder on this same disk.

Some operating systems, most notably Windows and earlier iterations of the Macintosh operating system, even let you create folders on the System Disk that aren’t earmarked for a purpose for you to use as your data folders. This also includes other programs keeping the user-created data in their own folders.

The secondary holding place for deleted files

Then there is the “Trashcan” or “Recycle Bin” folder which is used as a holding space for files you delete should you regret deleting them. When you delete a file from one of your folders on the main disk or other fixed disks in your computer, these files will end up in this “holding space”. Then if you want to remove them permanently, you have to delete them from this folder.

Removable Storage

USB memory key

USB storage device – an example of removable storage

All of the removable storage devices work on a freeform method of organising data across each of their logical volumes because there typically isn’t a requirement to keep certain folders for certain system processes.

This is except for memory cards associated with digital cameras because of the digital photography industry’s desire to implement a “Digital Camera File System”. Here, you have a DCIM folder for all digital-camera images and your camera will keep the pictures and videos you take in a subfolder of that DCIM folder, This was to simplify the searching process for digital images when you used a printer, photo-printing kiosk or electronic picture frame. There is also a MISC directory where DPOF print-order files are stored when you order photos to be printed using your camera’s control surface and either insert the camera card in to you multifunction printer or a photo-printing kiosk.

When you delete a file from removable storage, it is gone for good. As well, you need to make sure that you properly remove memory cards, USB memory keys and similar removable storage because most operating systems won’t write back all of the data changes to that storage device as they occur. Some operating systems like Windows allow you to immediately remove the classic floppy disks but most of them require you to use a “safely remove” or “eject” routine to properly write all the data to the removable medium before you can remove it. The Macintosh even allows you to drag the removable medium to the Trashcan to safely remove it.

Conclusion

The file system that your computer has is one of the key tenets of managing your data on your computer and it is about how your data is organised across multiple storage devices and within these storage devices.

Using FreeFileSync to sync media files out to your NAS

You use a regular Windows or Macintosh computer to curate your pictures, music and video files and store these files on your computer’s hard disk. Then you buy a high-capacity network-attached storage device to make these files available on your home network at all times and also as a backup or “offload” measure.

Normally this will require you to use Windows Explorer or Macintosh Finder to copy the files out to the NAS every time you synchronise them out to your NAS. This can be annoying especially if you have made changes to a few of the files or added a handful of files to the collection such as the latest downloaded images or a CD “rip”. Here, you have to answer a file-owerwrite prompt that the operating system puts up every time you write over an existing file as part of a copy process and this can be awkward if you did something like modify your files’ metadata or edited a photo, You could select the “Yes to all” prompts but this runs a slow copy process which transfers redundant data or work through each folder and file manually and find that you hadn’t reflected all the changes you had to reflect..

There is a free open-source application called “FreeFileSync” which automates the process of keeping your files that exist on two locations in sync.  This is available for Windows, Macintosh OS X and Linux and can work with locally-mounted drives or SMB network-shared folders.

Here, you can set up a “there-and-then” sync job or create a sync job affecting certain files and folders on both the source and destination in a particular way. A sync job that you save can affect multiple pairs of files and folders thus avoiding the need to create one job for each folder pair.

Prerequisites

FreeFileSync must be downloaded and installed on your computer

You download FreeFileSync from FOSSHub or Download.CNET.COM and install it as you would for downloaded software for your operating system.

Identify on your computer where your media manager software is storing your music, photos and videos.

Media libraries in Windows 8.1

Media libraries in Windows 8.1

In iTunes, this is found under the “Advanced” tab in the Preferences menu. Windows Media Player and Windows Live Photo Gallery use the Pictures and Music or “My Pictures” and “My Music” libraries created by Windows. Other media-management tools may use a particular folder that you set in their options or preferences window as the place for their media library.

CD rip location in Windows Media Player

CD rip location in Windows Media Player

Most audio-based media management tools like iTunes and Windows Media Player typically use the library as their import folder for when you “rip” a CD or purchase music through their online store whereas a lot of photo and video tools may have you create a separate import folder away from your library for images and video you import from your camera or scanner. This then allows you toe edit the images and video before adding it to your library.

Identify and make available the “media” folders that you are using to store your media on your NAS.

A NAS that uses a DLNA media server and an iTunes media server typically references a folder tree like “Media”, “Shared Media”, “Shared Music” or something similar. These are typically at the “Public” SMB mount point and are accessible using SMB/CIFS as well as these media servers.

If your NAS uses one shared media folder, create a sub-folder for the music files, another for the images and home video and another for other video like “download-to-own” content.

Create a media sync job

Setting up FreeFileSync for media syncing

Setting up FreeFileSync for media syncing

These actions are for a Windows computer and most NAS units

  1. Open FreeFileSync
  2. Click ProgramNew
  3. For each root folder representing your media collection kinds,
    a) Drag the root folder representing the media type on your computer to the left file list pane
    b) Drag the destination media folder for the media type on your NAS to the right file-list pane
    c) Click the + symbol to add extra media type pairs to your sync list.
  4. Click the gear icon next to the Synchronize button to determine the kind of synchronisation to take place
    In this case, you will have to select the “Update” option for this job. This effectively contributes new and modified files and folders that exist on the computer to the NAS without deleting any files that have been removed from the computer’s media folder. This is important if you just keep your files on your regular computer just to curate them before adding them to your media collection, or you “shift” older files to your high-capacity NAS to create space for newer files.
  5. Click on the “Update” button to select this option.
  6. Click on the “Save As” option to save this sync job as a file. Give it a name like “MediaSync” or “MediaNAS” to reflect the goal of it syncing your media to the NAS.

Manually running this sync job

Here, you open FreeFileSync, select the name of the “media sync” job and click “Synchronize” to start the sync process.

When to run this

Run the :FreeFileSync” job whenever you have done significant work on your media library like importing new media or editing existing media including the metadata. This can also be done as part of a backup routine before you start off the main data backup on your PC.

Formatting external hard drives to native file systems may allow them to work optimumly

Article

Using external drives to save space on your Mac? Format them first. | MacFixIt – CNET Reviews

My Comments

External hard disk

A typical external hard disk

This article highlighted the benefits of formatting external hard disks that are to be connected to an Apple Macintosh as HFS+ file-system disks. This advice hails well for formatting an external hard disk to an operating system’s native file-system such as NTFS for Windows or ext3 for Linux. This is because most of the external hard disks come preformatted to FAT32 more as an interchangeable file system that can be used on Windows, Macintosh and UNIX/Linux.

In the context of an external hard disk that you attach to a network-attached storage device for extra capacity, it may be worth looking through the instruction manual for your NAS or router to find the optimum file system. Some of the NAS units may work with ext3 or NTFS as the file system whereas most, including the routers with NAS functionality will stay with FAT32.

But this situation would apply to external hard disks that are primarily connected to the host computer all of the time rather than the “portable” hard disks that are likely to be moved between different computers and file systems. It wouldn’t hold well with thumb-drives and memory cards that you would expect to move between various computer devices including specific-purpose devices because most of these devices would only know the common FAT32 file-system.

This advice is best applied with an external hard disk that you have just purchased rather than one that contains all of your data because a format routine can wipe all of the data that exists on the hard disk.

Using your smartphone or tablet with your network-attached storage

Why use your smartphone or tablet with your NAS

There are times when you want to upload or download files from your network-attached storage device using your smartphone or tablet computer. Examples of this include offloading files from a low-capacity device, through making media that you took or acquired through your device available at all times from your DLNA-equipped NAS to simply backing up data held on your device.

Of course, you may simply treat that NAS simply as a network transfer point for your data. Examples of this may include working with documents that you start on an iPad and want to complete on your regular computer or conceptual “rough-shot” pictures that you take you your phone’s camera and want to work with further using Photoshop and other software on your computer.

How is it done

But how am I able to do this with my network-attached storage and my mobile devices? Some network-attached storage devices may use a Web front for the file collections where as others may implement certain extensions to DLNA for uploading and downloading some media files. This latter situation is infact a long-term goal for the DLNA Home Media Network, especially when it comes to shifting or syncing multimedia files like music or images.

These environments don’t necessarily provide a consistent or ideal user experience for the mobile device user. This can typically be due to a Web front that is optimised for desktop use only to DLNA server and client apps not offering the proper sync or file-transfer functionality.

SMB file-manager apps

The preferred method that I would use is SMB/CIFS network file handling which every network-attached storage device supports thanks to Linux’s SAMBA software. Even the USB-linked file servers that are an increasing part of high-end routers like the Freebox units do support SMB as well as the Internet HTTP and FTP file transfer protocols. This has been a standard for regular computing devices with the Microsoft Windows Platform since Windows For Workgroups 3.11, then was exposed to Linux regular computers through SAMBA and has been exposed to the Apple Macintosh platform since MacOS X.

The platform-based mobile devices now can join the SMB party through the use of SMB-enabled file-manager apps. These are typically low-cost or free apps that expose the mobile device’s file system and the SMB file shares (entry points) made available by computers or network-attached storages. Some of them have file-viewer functionality for file types not supported by your device’s file handlers.

iOS

Intuitive Commander (App Store – $0.99)

FileBrowser (App Store – $4.49)

Syncsellence (App Store – $5.49, free limited version App Store)

Android

ES File Explorer – I use this on my phone (Android Market)

File Expert (Android Market)

File Manager (Android Market) / File Manager HD (Android Market) – Rhythm Software

Blackberry

File Expert (Blackberry App World – US$1.99)

File Manager Pro (Blackberry App World – US$4.99) – Terra Mobility

ArrangeIt File Manager (Blackberry App World – US$1.99) – Conceptual Designs

The various app stores for the popular mobile-device platforms will list more of the file manager apps with SMB file transfer and you can find them using the terms “SMB file transfer” in your search query.

It is also worth noting that your NAS’s vendor may offer file-transfer apps for their device on the iOS and/or Android platforms so you can transfer the files to their device. These programs may also work with the remote-access functionality that some of the consumer and SMB NAS units provide, thus keeping login credentials for the devices and streamlining the remote-access experience.

Other issues worth highlighting

iTunes-purchased content

You may have problems copying content that you purchased with iTunes on your iPhone or iPad directly to the NAS due to Apple’s setup for these devices. But they have improved the iTunes and iOS setup to allow a user to download the purchased content to an instance of iTunes run on a regular computer even though they purchased it on the iOS device. This works best if the regular computer’s iTunes library is referencing the NAS in question.

People who use iOS platform devices that aren’t updated to iOS 5 will need to tether the device to their iTunes-enabled regular computer. Then they will need to use the “Transfer Purchased Content” option in iTunes to copy the content they bought on the device to the regular computer or NAS.

It will also be important to make sure that audio content is downloaded as MP3 files rather than protected M4A files.

One way that Apple can work this situation out better is to implement read-write ability to iTunes (DAAP) servers for the iPod media-management app in their iOS platform. Here, the software could then support improved “offload” functionality. This may not come about due to Apple’s investment in and their fanbois’ preference for the iCloud as a large-capacity storage service.  But practically-minded Apple enthusiasts could place more value on a NAS as an extra-capacity data store so they know where their iTunes content is all the time.

File-transfer operating conditions

When you transfer files between your mobile devices and the NAS, make sure that you have a strong Wi-Fi signal at your mobile device and that the device has sufficient battery strength. This could be achieved through having the device connected to its charger while the transfer goes ahead.

NAS setup conditions

If you are transferring media files to the NAS, you would need to transfer them to the media folders that are referenced by the media-server software on that device. This may be made easier by using the file manager software’s “bookmark” or “favourites” options to point to the start of the NAS’s media folder tree.

It is also worth keeping other personal and workgroup shares on the NAS simply for backup or transfer purposes and referencing these with your file-manager app.

Conclusion

Once you are able to know that you can use the SMB file transfer method for moving data between your NAS, tablet and smartphone, you can see more value out of these mobile-computing devices.