Photoshop selects pixels based one of the three properties, as shown in the image below:

1. Chroma: is the color of the pixel. Color of a pixel is the level of RGB values and color based select tools like the Magic Wand. Select by color uses the RGB% as a criteria to group pixels based on the set tolerance values.
2. Luma: is a selection based on Illumination levels. Illumination levels are whiteness of the image, which is nothing but higher values for all the three R, G and B channels. Photoshop doesn’t have a native interface for a Luma based selection, but many existing tools can be hacked to get a similar result.
3. Masks: are based on spatial position. Pixels are selected and discarded or masked by a parallel layer bound to corresponding image layer. Masks are grayscale and brightness of the mask is called an Alpha level, which is the degree of opaqueness of the corresponding layer pixel. It’s also called the Transparency/Opacity channel as in RGBA etc.

A Note on Masks

In this article, words like Masks and Alpha will be used interchangeably. The process of transferring images to layer masks will be shown once and then used repeatedly without explicitly mentioning thereafter.

Step 1

Open a new image that you want to use as a mask. Double-click the background layer and create a new layer, which is “Layer 0.” Make a new layer, which is “Layer 1,” on top of it and fill with a uniform color, then hide “Layer 1.”

Step 2

Go to “Layer 0″ and click Select > Select All (Command + A) to select all pixels. Next, click Edit > Copy (Command + C) to copy all pixels to the clipboard.

Step 3

Go to “Layer 1″ and create an empty Layer Mask. Initially an all transparent Layer Mask will be created. Alt-click the layer mask to see it in the canvas.

Step 4

With the Layer Mask visible, paste onto the canvas by clicking Edit > Paste (Command + V). A grayscale version of the copied image will be pasted in as the layer mask.

Step 5

Click on the fill layer thumbnail (not mask) to see the result. Any image can be applied as a Layer Mask to another layer. The opacity of the layer will be controlled by grayscale level of the mask.

More on Masks

The easiest way of creating a Layer Mask is to click the Add Layer Mask button with something selected on the canvas. The selection will be saved as a layer mask attached to the current layer and everything outside the selection marquee will be masked and hidden.

Activate Layer Masks

Alt-clicking on the Layer Mask makes it active for editing. Only activated layer masks can be edited which are represented by a thin white outline around them.

Add Shapes to Layer Masks

Dragging any shape onto the canvas while a layer mask has been activated will draw it on the Mask instead of the layer itself. To draw it again on the layer we need to deactivate the Mask by clicking on the layer thumbnail.

Add Brush Strokes to Layer Masks

The same is true for brush strokes. Anything painted on the canvas with a Layer Mask selected will draw it on the Mask instead of the Layer. We can use this property to select objects with irregular boundaries from backgrounds, by painting everything else black on the Layer Mask. White brush strokes will reveal and blacks will conceal.

Copying a Layer Mask to Another Layer

Finally, we can transfer the Layer Mask to another layers by simply loading it as a selection (Command-click) and unloading as we did in the first step by clicking the Add Layer Mask button. Or by Alt-dragging the mask to another Layer (Not Shown).