Vertus Fluid Mask 2.0 Manual for Mac · Vertus’ Fluid Mask software is a set of advanced paint tools that cut out highly complex images with absolute precision. Designed to make

Vertus’ Fluid Mask 2.0 Manual for Mac

© Heligon 2006

Vertus’ Fluid Mask 2.0

© Heligon 2006

Manual for Mac

Introduction to Fluid Mask Manual Vertus’ Fluid Mask software is a set of advanced paint tools that cut out highly complex images with absolute precision. Designed to make life easier for everyone who needs to mask images, it offers an intuitive, accurate and fast approach to cut-outs.

The software automatically detects even hard-to-see edges within the image and pours the mask right out to each edge, giving the user much greater control over the cut-out. Using Vertus’ Fluid Mask 2.0 you can display every edge that Fluid Mask can detect within the image by activating the “Edge Overlay. You then apply a mask by simply selecting image objects displayed in the Edge Overlay view – similar to coloring by numbers.

Fluid Mask 2.0 uses breakthrough technology which mimics the way the eye, optic-nerve and brain perform visual processing. When opening Fluid Mask, the software automatically makes an analysis of the image metadata about texture change, edge and color information within the image. This is used to locate edges when painting masks and blending edges.

You can see the benefits of this approach when you try these:

• Hair and fur - Try cutting out lattices

• Speed – Use “One-MaskTM”

• Control – Invoke the “Region Editor” for better color selections

• Quality – Use “Auto-Complex” for automatic edge characterization and complex masking

• Ease of use – Checkout the intuitive tools set and the “Edge Overlay”


© Heligon 2006

ContentsThis Manual covers:

• How to get started – Download and installation

• An introduction to the product – Tools and features

• Using the product – Applying masks, adjusting the “Edge Detection Options”, using regions, forcing an edge

Throughout the Manual you will see - Top Tips signs. These offer useful tips and alternative ways of doing things that will help you to create great cut-outs.


© Heligon 2006

Manual for Mac

Getting started

DownloadPlease download Fluid Mask athttp://www.vertustech.com/FM_download.htm

InstallationDownload the installer file for Mac from the link above and save it to your computer and start the installation wizard. Follow the on-screen instructions to complete the installation. Fluid Mask will be automatically installed into the Plug-ins folder of Photoshop.

Fluid Mask is started through Adobe Photoshop. You can find the link to start the software in Photoshop under > Filter > Vertus > Fluid Mask. When you open Fluid Mask for the first time you are prompted to make the choice between running Fluid Mask as a demo (limited functionality) or activating it fully using your Serial Key. If you choose to open it as a demo you may later enter your Serial Key to activate it fully.

Please note:For non-English versions of Photoshop the name of the Plug-ins folder is localized into the native language. The automatic installation process does not take account of this. Instead it will create an English Plug-ins folder and install in to that.

Please use the following way of installation to make Fluid Mask available in the menu > Filter in Photoshop: When the installation process starts, please use the manual install option in the installer and select the localized Plug-ins folder inside your Photoshop application folder.Then continue and complete the installation process.

Activating Fluid Mask with your new Serial Key

Registering Fluid Mask at First Use

If this is the first time you have run Fluid Mask from download, the registration wizard will appear automatically when you load your first image from Photoshop and invoke the Fluid Mask plug-in from the Filter menu on the toolbar.

1. Make sure you are connected to the internet.2. Simply follow the on-screen instructions to activate the full version of Fluid Mask remembering not to indicate that you want to run the software in Demo mode.

Registering from Demo Mode

If you are running Fluid Mask in Demo mode, to register the product and unlock the full version you should:

1. Make sure that you are connected to the internet.

2. Click “Photoshop” from the toolbar and from the drop down menu select “About Fluid Mask”

3. Click the button marked “Register” and the registration wizard will appear

4. Follow the on-screen instructions to activate the full version of Fluid Mask


© Heligon 2006

A tour of the product featuresThe first part of this Manual is to take you through the tools, tabs and options to make you familiar with the product features. These are also available to you as a Quick Guide from “Manuals and Tutorials” on our website at:http://www.vertustech.com/fm_tutorials.htm

It may be useful to print off the Quick Guide as a reference when first starting Fluid Mask and when looking through the rest of this Manual. The Quick Guide also shows short-cut key strokes which are not included here.

View tabs There are three convenient view tabs in Fluid Mask: Source, Workspace and Output which make it easy to follow the cut-out process, from start to finish. The Source tab contains the original image. The Workspace tab is where you apply masks. The tab “Output” tab contain the cut-out that you created.

Source – original image

Workspace – mask application

Output – final cut-out

There is three types of mask may be manually or automatically applied to an image.

Delete mask – by default indicated by red, masks the area to be deleted

Keep mask – by default indicated by green, masks the area that you want to keep

Complex mask – here in blue, the complex mask comprises mixed Keep and Delete pixels at the edge of the object (this mask is added automatically when the other two are applied)

Edge overlay – to display the edges Fluid Mask finds within the image

Max opacity – to adjust the level of all mask opacities


© Heligon 2006

Manual for Mac

Exact pencil – masks only the pixels you paint

Global brush – covers all the pixels across the whole image that are similar to the colors and texture of the pixels you paint

Eraser – removes mask

Smooth – smoothes jagged edges of mask

Force edge – forces a mask barrier when Local Brush doesn’t indicate an edge

Select region – selects and moves Regions

Pan – lets you move the around your image with ease

Colours – the colors are used to indicate Keep, Delete and Complex masks

Local brush – masks all the pixels of a similar color in the immediate area that match the pixels you paint

Complex – exact pencil, local and global brushes to apply complex mask

Clean – cleans up speckles (holes) within mask selections already made

Test render – previews sections of the cut out before performinga full cut out Continually drag marquee over area of interest as you work for quick snapshots of your work in progress – gives you confidence in your masking and ensures no upsets at Create Cut-Out stage

Create region – (see later) selects an area of the image requiring detailed work

Zoom – Zooms in and out of the image when working on fine detail

Background toggle – alters the appearance of backgrounds:Transparent – the background as the traditional Photoshop checkered background Color – shows the background as a specific color Double click on the color square to alter the color background. Use heavily contrasting background to pick out any pixels that may have been missed in the masking process.

Create cut-out– creates a final cut-out


© Heligon 2006

Tool controlsFor most tools in the main tool box of Fluid Mask, the “Tool Options” window will come up. This offers options to adjust variables such as the brush size or strength. The “Tool Options” are available for most tools in the main tool box. The sample below refers to Local Mask Brush controls.

NavigatorAs with Photoshop, you can use the Navigator panel to zoom and move about your image.

Brush size – adjusts the pixel size of the selected brush

Brush strength – adjusts the amount of mask the selected brush will apply

Auto complex – (see later) adjusts the relative amount of automatically applied Complex mask or turns the feature off

Auto-smooth – smoothes jagged edges of mask automatically

Include masks – appliesnew mask to colors already assigned to a mask

Image – thumbnail of your image

Window ratio – red boarder indicates the section of image visible in the image window

Zoom – zooms in and out of the image using the zoom slider


© Heligon 2006

Manual for Mac

Edge Detection OptionsThe “Edge Detection Options” Panel offers various settings that control the masking process in Fluid Mask. The adjustments you make in the “Edge Detection Options” panel allow you to refine the information about an image that Fluid Mask uses to detect edges and find objects within the image. The results of the adjustments you make are new object boundaries. The image objects and their boundaries are then visible in the “Edge Overlay” view. The “Edge overlay” view can be activated/deactivated in the main window of Fluid Mask through activating the check box at the bottom (see screenshot page 6).

Edge sensitivity – Increase the sensitivity to find more edges within the image. Increase to maximum and every pixel becomes an edge. Switch off the “Edge overlay” and adjust brush strength to control the size of your selections.

In-Focus edge width – here you can specify the width (in pixels) of an edge between two objects that are in sharp focus. Zoom right in on the image to see how wide the edges really are. In large images it can be 10’s of pixels. Reduce the width to find more edges in the image.

Workspace resolution – To make masking faster the source image can be downscaled. You can set the number of source pixels to workspace pixels ratio here. The output tab always remains at full resolution so that the cut-out generated does not loose any detail.

Test render – specifies how much difference in color there is across edges in the image.

Low - helps to find edges between objects of very similar color.

High- is useful to speed up cutting out simple images by reducing the number of edges.

Texture filter – controls the texture analysis.

Fine – helps to accurately select many small objcts (e.g. leaves against sky).

Coarse – is useful to prevent detection of lots of tiny objects (e.g. blades of grass) by treating those areas as uniform patches of texture.


© Heligon 2006

Region EditorThe Region Editor is used for fine tuning and complex detail (See below). It allows you to move colors within a selected region from one mask to another, or to assign unallocated colors to one of the masks. This is helpful in case you applied a keep or delete mask to an image but some pixels of certain color (around a blurred edge, for example) were marked by the wrong mask.

The tabs “Keep”, “Delete”, “Complex” and “Unassigned” will show you to which mask the color in the region you marked has been assigned.

Select the Region Tool (Rectangular or Polygonal selection tool) from the Basic tool palette to open the

Assigned color palette – colors within a region that have been assigned to a mask type

Unnasigned color palette – all colors within a region that have not yet been assigned to a mask type

Sort colors – organizes the colors into 1D or 2D views

Selected colors – colors selected within the Region Editor and waiting to be assigned to a mask type

Region color palette – colors within the selected region

Number of colors – reduce or expand the color detail when working with large regions

Turn on/off Auto-Complex within a region – turn off Auto-Complex to prevent automatic movement of mask assignments

Assign to complex mask – apply Complex mask to selected colors

Assign to keep mask – apply Keep mask to selected colors

Assign to delete mask – apply Delete mask to selected colors


© Heligon 2006

Manual for Mac

Using the product This section looks at how to use the product concentrating on several key areas:

• Making mask selections and creating a cut-out

• Adjusting the Edge Detection Options to improve masking of difficult images

• Using Regions for detailed color based selections

• Forcing an edge as a last resort


© Heligon 2006

Making mask selections and creating a cut-outThe key to making a good cut-out is to make good mask selections. If the selection is wrong the edge blending will be wrong.

For the first few times you use Fluid Mask, experiment with different selections and settings for your brushes and use “Test Render” tool from the main tool box to review the results. You can use Test Render even if you only apply One-Mask (see Quick Cutting in the Tutorial)

Making mask selectionsTo make any cut-out you need to apply three different mask types:

• Delete mask in red to mask all those areas of the image to be deleted (or background areas)

• Keep mask in green to mask all those areas of the image to be kept (or foreground areas)

• Complex mask in blue to mask the edge areas contaminated by foreground and background colors

Note: these representative colors may be changed using Fluid Mask preferences in the menu > Edit > Preferences. However in this Manual the masks Delete, Keep and Complex are always indicated using red, green and blue.

There are two paths you may take to cut out images.

• The first is quick cutting using One-Mask™. Paint all of either the Keep or Delete mask and then press the button “Create Cut-Out” in the main tool box. Fluid Mask then Auto- Fills the remaining two masks and cuts the object out. This is the quickest and simplest way of making a cut-out.

Using One-Mask™ makes sure that only one of Keep or Delete masks have been painted otherwise the operation will not auto-fill and revert to three mask cutting

Alter the amount of automatically applied Complex mask applied in the auto-fill process by adjusting the auto-complex settings in the brush tool options

You may separate the auto-fill process from Create Cut-Out using Auto-Fill Image from the Image Menu

• The second path is by painting the Keep and Delete Masks. It’s a slower process but you have more control over the masking. When Keep and Delete masks are painted close together at an edge Complex Mask is automatically created (see Auto-Complex mask below).

Auto-Complex maskingAuto-Complex masking paints Complex Mask over the edge between Keep and Delete masks and adjusts the position of the Keep and Delete masks automatically. The thickness of the Complex mask and position of the other masks is determined by the character of the edge. A thin Complex mask will apply to hard edges, a thick Complex mask should be used for defocused edges. Further, the relative amount of Complex Mask may be adjusted using the Auto-Complex settings (see screenshot page 14). The Auto-complex Masking can also be turned off.


© Heligon 2006

Manual for Mac

Don’t fall into the trap of changing the overall thickness of the Auto-Complex masking to alter the mask in one area; otherwise you’ll throw out all your masks. These kinds of “local” changes need to be made by reverting to the Workspace, manually adjusting the masks and invoking Create Cut-Out again or by altering the Auto-Complex settings within the Region EditorTo apply mask selections you use Fluid Mask’s unique brush set.

Painting mask using brushesThere are three brush types to apply mask in three different ways.

Exact Applies mask to only the pixels you paint.

Local Masks all the pixels of a similar color and texture in the immediate area that match the pixels you paint.


© Heligon 2006

Global Covers all the pixels across the whole image that are similar to the colors and texture of the pixels you paint.

Brush optionsEach brush type has a number of options that can be adjusted to help your masking:

Adjust brush size (common to all brush types)Adjust the size of your brush by pixel adjustments.

Adjust brush strengthBrush strength determines how far your local mask selections will “grow”. Use it to determine the correct strength for the image and task.

When the brush strength is at its minimum the local brush acts just like a flood fill going right up to the nearest edge within the image. Turn on the check box “Show Edges” (Edge Overlay) at the bottom of the main window in Fluid Mask to see exactly the edges in the image have been detected by Fluid Mask.

Holding down Ctrl when you release the mouse at the end of painting a mask gives you a weaker brush. This can save you from having to change brush strength too often, and makes working with hair and complex areas containing many colors easier.

To accurately find edges of very complex shapes such as hair or fur, set the edge sensitivity to maximum (20) in the Edge Detection Options panel and click Apply. This turns every pixel in to an edge (you will need to turn off “Show Edges” by pressing F2). Now rely on the brush strength to control how large your selections are. This is particularly useful with the Global brush for selecting complex lattices such as trees.

Auto comples controls


© Heligon 2006

Manual for Mac

Re-masking – making mask selections which overwrite areas already masked (common to all brush types)By default you may only mask areas which have not already been masked (unassigned areas). This may be over-ruled to allow you to overwrite existing mask selections by checking the check boxes in the Tool Options of each brush by selecting Include Keep, Include Delete or Include Complex.

Invoke Auto-Complex (common to all brush types and Test render)The width of the automatically applied Complex Mask (see above) is dependent on the character of the edge but can be adjusted into: None (turns Auto-Complex off) –Thin – Medium (default) – Thick.

Invoke Auto-Smooth (Local and Global brush only)Automatically performs the smooth function as a mask is applied. (see Smoothing a mask). The amount of smoothing is controlled from the smooth tool options.

Smoothing a maskUse the Smooth Picker in the main tool box to smooth jagged edges of the selected mask. The smooth picker’s strength can be adjusted using the slider in the Tools Option panel. Previous selections may also be smoothed from the Image Menu.

Cleaning a maskUse the Clean Picker to remove small holes from within the selected mask. The Cleaning Picker’s strength may be adjusted using the slider in the Tools Option panel. Additional clean operations may be performed from the Image Menu.


© Heligon 2006

Adjusting the Edge Detection Options to improve masking of difficult imagesThere are a number of options that control the image analysis found in the Edge Detection Options panel. When loading an image into Fluid Maks, the program attempts to estimate sensible default values for an image. However, due to the huge range of images it is sometimes necessary to adjust these values manually to achieve a better result in edge detection.

The Edge Overlay shows you clearly what edges have been found and whether you will need to adjust the values to help you mask the image.

It is worth spending time getting to understand how changing the different options will affect the edges found. The Edge Overlay is there to help you.

The mask you have already laid down is preserved when the Edge Detection Options are changed. This allows you to work on each part of the image using the most appropriate values for that area by changing the settings as you go along.

After changing the value of the options please click on “Apply” in the Edge Detection Options panel to start the re-analysis of the image.

The button “Revert” restores the values of the options most recently used to analyze the image.

The button “Defaults” restores the default values for this image.

Use F3 to change the style of the edge overlay (here to white) so that the edges are clearly visible against the colors of the image. Use F2 to turn the overlay (check box “Show edges”) on and off.


© Heligon 2006

Manual for Mac

Adjusting Workspace ResolutionFluid Mask measures your computer’s performance and ensures that the Work-space resolution used allows for acceptably quick mask selections (large images tend to slow Fluid Mask down at full resolution). It does not affect your image as the mask resolution is subsequently matched to your image at the Create Cut-Out stage.

Adjustments to this automatic process may be made in the main menu under > Image > Preferences. You may either set the automated resolution setting for Speed of Selection (lower resolution and the default) or Greater Detail (higher resolution).

This automatic process may be over-ridden in the Image menu or the Edge Detection Options panel. Primary reasons for undertaking this are:

• Improving masking around very wide defocused edges (reduce image resolution)

• Improving edge selection where colors are very similar either side of the edge (increase image resolution)

• Reducing image noise (reduce image resolution)

• Improving detail in the Workspace (increase image resolution)

When the image is first loaded, try masking and using the Test Render tool on the difficult areas first to see if the masking is accurate and preserves enough detail at the edges.

Please note that changing resolution has to remove any mask you have already laid down and it is not possible to restore the lost mask unless you save the workspace from the File menu. Fluid mask will warn you of this before changing resolution.

Adjusting Edge SensitivityThe most common and obvious adjustment you will want to make is to the edge sensitivity. To find more edges in the image simply increase it. To find less edges decrease it.

For simple images turn down the edge sensitivity until just before the edges you want to keep disappear. This will show a minimum of objects to mask within the image and will allow you to apply the appropriate masks faster.

For complicated images with lots of detail, such as trees, grass, hair, feathers or fur it is often necessary to turn up the edge sensitivity a long way. This can make the edge overlay hard to use (because it obscures most of your image) and can make laying down mask slow because only small areas of the image are selected at a time.

To overcome this turn off the edge overlay and increase the brush strength. When the brush strength is increased the local and global brushes will select more at a time, flooding into areas of similar color and texture to the area you paint.

For cutting out complex lattices such as trees against sky turn the edge sensitivity up to maximum and find the appropriate brush strength to use with the global brush.

If only Edge Sensitivity is changed the analysis of the image is faster than if any of the other Edge Detection Options are changed.


© Heligon 2006

Adjusting In-Focus Edge Width

The In-Focus Edge Width is one of the most important options in order to improve the results of the edge detection. The width of an edge between two objects depends on how in or out of focus the objects are and how large the image is.

Fluid Mask attempts to estimate the In-Focus Edge Width based on the size of the image. However this is often incorrect since the image may be just a zoomed in part of a larger image. It would be hard to determine this automatically.To work out what the in-focus edge with is, zoom right in on the image until you can count the number of pixels wide across a typical edge.

By adjusting In-Focus Edge Width you can find more or less detail in the image. To get rid of fine detail increase the edge width, or to find more detail decrease it. Beware of decreasing it unnecessarily far for large images since it will slow the analysis down and require a great deal of memory.

Adjusting Edge Contrast

A low contrast image is one in which background and foreground colors are very similar. For example an image shot in poor lighting conditions. Even if the lighting conditions are good, it can be hard to detect the boundary between dark objects and their shadows (such as tires of a car) because the contrast across the edge is very low.

In such situations setting the Edge Contrast to Low will help at the expense of a slightly slower analysis. In general this will increase the number of edges found in the image.

Alternatively for high contrast images it can be faster to work with the image by increasing the Edge Contrast to high. This will tend to find less edges in the image.

Adjusting Texture Filter

A powerful part of Fluid Mask’s image analysis is its texture filter. You can set the relative size of repeating patterns in the image that will be considered as uniform areas of texture.

Images of natural scenes, for example, often contain areas of grass of leaves. To preserve the fine detail of individual leafs or blades of grass set the Texture Filter to Fine. On the other hand to amalgamate all these small objects into uniform areas of texture that can be quickly selected in one go, change the Texture Filter to Coarse.


© Heligon 2006

Manual for Mac

Use Regions for detailed color selectionsRegions are an alternative means to apply a mask. Rather than mask by selecting pixels, as with the brush tools, the Region Tool allows you to mask areas by selecting colors.

In addition, regions may be used to constrain the effects of a brush tool. If a brush selection is started within a region, its effects will be confined to the area covered by the region.

Creating Regions There are two forms of Region created by different tools:

Click on the desired Region type and draw the region in the Workspace. When drawn the region, the “Region Editor” will automatically appear, containing all the colors in that region. The colors will be shown in the three different tabs “Keep”, “Delete” or “Complex” depending on what mask has been applied to them before, or under the tab “Unassigned” when no mask has been applied to those colors yet.

It is possible to create as many regions as needed.

Use the Select Region Tool to

• activate an existing Region. Double click within the Region for it to maximize to the full size of the Workspace window

• adjust a Region’s size by pulling at the corners

• move a Region


© Heligon 2006

Using the Region Editor

When a Region is created or selected, a palette of the colors of all pixels within the Region is created in a Region Editor. These are sorted by mask status.

By highlighting colors in the Region Editor, corresponding pixels in the Region are highlighted in yellow.

These selections may be altered to another mask status by clicking on the colored buckets at the bottom of the Region Editor. Clicking on the green bucket assigns the selected colors to the Keep mask, the red to the Delete mask and the Blue to the Complex mask.

Select a region – drag a marquee over the area to be worked on

Region editor appears – the Region Editor is automatically launched displaying all the colors in the selected region

Select one or many colors – colors can be selected in groups or individually within the Region Editor

Selected colors indicated – selected colors are high-lighted on the image


© Heligon 2006

Manual for Mac

These selections may be altered to another mask status by clicking on the colored buckets at the bottom of the Region Editor. Clicking on the green bucket assigns the selected colors to the Keep mask, the red to the Delete mask and the Blue to the Complex mask.

Experiment with the Number of Colors slider to find the optimum palette management, enabling you to make great selections

Using Regions to constrain actions

Another feature of the Region is that it acts as a boundary to certain actions:

• A Region will constrain any selection (Exact, Local or Global) made within it.

• Auto-Complex Masking may also be turned off within a Region but still be applicable elsewhere.

Forcing an edge as a last resortSometimes it’s necessary to mark an edge. This might be because the edge is not visible, or the Local Brush cannot “find” it despite the adjustments made above.

To do this we use the Force Edge tool

Assign colors to mask type – apply mask by assigning selected colors to the correct mask “bucket” – in this case the selected color is assigned to Delete

Mask then appears in workspace – Here, as the selected colors are assigned to the Delete mask, the selected pixels change from Yellow to Red in the Workspace

Tab views in the Region Editor – view colors within the Region assigned to each mask type and those colors still unassigned using the Tabs at the top of the Region Editor


© Heligon 2006

The Force Edge tool creates an explicit edge between two areas preventing subsequent Local Brush strokes from “leaking” across the edge. The edge line includes anchor points which can be picked-up and moved using the Select Region pointer tool.

Next stepsTry the step-by-step tutorial on some of our example images or test Fluid Mask on an image of your own.

For other hints, tips and problem solving visit our website at www.vertustech.com and go to the Cut-out Gallery. There you’ll find examples of great cut-outs and advice on how they were achieved, all submitted by Fluid Mask users. More support from the team at Vertus and other Fluid Mask users can be found in our Vertus’ community forum. You can reach the fo rum through our web site.

Documents

Vertus Fluid Mask 2.0 Manual for Mac · Vertus’ Fluid Mask software is a set of advanced paint tools that cut out highly complex images with absolute precision. Designed to make