USER GUIDE

Version 2.2

Issue 2

This
document contains proprietary information of IVIM Inc,

and is not to be disclosed or used except
in accordance with applicable contracts or agreements.

Copyright©2014
IVIM Inc. All Rights Reserved.

www.nouse.ca

Nouse – User Guide

Version 2.2

Issued – June 2014

The following third-party products are used by Nouse
Applications Suite:

·
IBM is a registered trademark of
International Business Machines Corporation.

·
Java is a trademark of Sun Microsystems,
Inc.

Microsoft,
Visual Studio, Visual C++, Windows, Vista, Windows XP, and Windows 7 are
registered trademarks of Microsoft Corporation.

·
OpenCV Toolbox is a registered trademark
of Intel Corporation.

·
All other product names are trademarks or registered
trademarks of their respective companies.

All
rights reserved. This document may not, in whole or in part, be copied,
photocopied, reproduced, translated, or converted to any electronic or
machine-readable form without the prior written consent of IVIM Inc.

Nouse
is a trademark of IVIM Inc.

Table of Contents

Preface. 4

Chapter 1 – Welcome to Nouse. 5

Chapter 2 – Getting Started. 6

Chapter 3 – Nouse Basics. 7

3.1 NouseCursor 7

3.2 Nouse Navigation Modes. 9

3.2.1 Joystick Navigation Mode (Default Mode) 9

3.2.2 Direct Navigation Mode. 9

3.3 Nouse Clicking Mode. 10

3.4 Nouse Interface. 12

3.4.1 Camera Feed. 12

3.4.2 Pause Button. 12

3.4.3 NouseBoard Button. 13

3.4.4 Recalibrate Button. 13

3.4.5 Settings Button. 13

3.4.6 Disable/Enable Button. 13

3.4.7 Exit Button. 13

3.5 NouseBoard Mode. 14

3.5.1 Using NouseBoard. 14

3.5.2 Cell Selection. 15

3.5.3 Letter/Option Selection. 15

3.5.4 NousePad Text Paste. 16

3.5.5 Editing Text 16

3.6 Voice Commands. 18

Preface

Purpose

Nouse
or “nose as a mouse” is a technology that enables users to
control the computer mouse pointing device hands-free using only a webcam as an
input device and the movement of the user’s nose. The
unique, patented technology uses
advanced video recognition algorithms to map the movement of the user’s nose to
the movement of a computer mouse device thereby allowing a user to operate a
computer hands-free. This User Guide will navigate the user through the
use of the technology and provide information on its various features and
functionality.

Audience

Nouse application was developed for anybody to use, however,
it will be most useful for those computer users who have difficulty using their
hands to manipulate a mouse or keyboard, or for those who want to take breaks
from using their hands due to fatigue/injury. The solution is targeted for an older
child or adult whose physical limitations (strength, range of motion) make
using their hands/arms for operating a computer difficult or fatiguing.

The ideal Nouse user needs to have good control of their
neck movement, as well as an ability to learn new computer skills. Their vision
(with or without glasses) must be sufficient to see and follow the cursor
around the screen.  Nouse does not create
more eye fatigue than standard computer use.

A Nouse user does not need to have full neck range of
motion, as Nouse can be operated by very small neck movements in all 4
directions. Nouse is well suited to
people with cervical spinal cord injuries, ALS or brain-stem stroke. A person
with a slight tremor could use Nouse when settings and targets are adjusted to
accommodate for their condition, however a person with significant tremor or
spasticity would likely not be a suitable candidate.

Computer experience would be an advantage, but not at all
necessary to learn how to use Nouse.

This User Guide assumes that the
user has a basic knowledge and understanding of Windows applications, Web
browsers and concepts such as double-clicking, menu bars, and dialog boxes.

Graphics

The appearance of graphics found in
this document may vary according to the site configuration and operating system
used.

Related
Documentation

The user should use the Nouse Quick
Start Guide, Nouse Settings Guide and the Training video in conjunction with
this Nouse User Guide.

Chapter 1 – Welcome to Nouse

The ability to use computer technology is almost mandatory
in today’s society. Many aspects of everyday life, including schooling,
shopping, banking, business, personal and employment require interaction with a
computer. For those who find working with a physical keyboard and mouse
cumbersome or have a disability due to disease or injury that prevents them
from using a computer, have lost one of the most important communication
mediums in the world today. This limits their ability to socialize, to entertain,
to work, to use the advantages of the internet and to feel they are a part of
society.

As computers become more and more integrated into our daily
lives, we need to ensure that Assistive Technology (AT) is there to allow
people with an arm or hand disabilities to be able to operate computers. A
growing area of AT is hands-free computer technology that allows users with a
disability to control a computer mouse and keyboard using other parts of their
body such as the nose. Using the movement of other parts of your body can provide
a different method for executing common tasks, such as pointing, clicking, and
typing.

Nouse
or “nose as a mouse” is a technology that enables users to
control a computer hands-free using only a webcam as an input device and the
movement of their nose. The unique, patented technology uses advanced video
recognition algorithms to track the user’s nose and map the movement to the
mouse pointing device of a computer. This allows a computer user to operate a
computer hands-free using their nose.

Essentially, Nouse is recognized as the first technology to use the nose as the principle
feature for facial tracking, a process that involves setting up a webcam on a
computer equipped with tracking software. The computer registers an image of
your nose and then maps it onto the on-screen cursor, which you control with
the movement of your nose. The nose's convex shape enables the computer to
track movements with dramatically improved precision over traditional methods.
Even if you turn your head to either side, the nose will always be within view
of the camera, unlike the eyebrows or the mouth, which were formerly used for
facial tracking.

Unlike other systems currently available to individuals who
cannot easily access a computer using their hands, Nouse requires only Nouse software
and a webcam. This makes Nouse extremely attractive as it is more affordable
and portable than current systems.

The key functions of
Nouse application include navigation of a computer mouse pointer (i.e., cursor movement)
and mouse actions (e.g. click), voice recognition, and keyboard operations.

Chapter 2 – Getting Started

Before getting started, the user must first install Nouse
application. The Quick Start document is designed to assist the user in installing
and getting the application running quickly. To optimize the use of these
guides the user should take the following steps to getting started:

a)      Refer
to the Quick Start Guide for installation

b)      Refer
to this User Guide for an overview of the basic features

c)      Refer
to the Settings Guide to customize Nouse application to suit the user’s unique
setup needs.

Chapter 3 – Nouse Basics

This chapter provides an overview of the basic features of Nouse
application.

NOTE: This
chapter assumes the use has already installed Nouse application on their
computer.

3.1 NouseCursor

The main component of Nouse is NouseCursor (shown in Figure
3.1). NouseCursor is the equivalent of a standard computer mouse arrow icon. It
is the pointing device that tells the computer what the user is looking to do
on the screen.

Cross-Hair

Background

Purple
Pointer

Red
Circle

Text Area

NouseCursor appears on the screen as a square box and consists of six
(6) components designed to provide the user with feedback on what action is
being executed.

Figure 3.1: NouseCursor
in Joystick mode

The following provides a description of each component of NouseCursor:

1)      The purple pointing component – This
can be found in the top left hand corner of NouseCursor. The computer uses the
top left hand corner as the pointer. Pointing to an item on the screen means moving your
NouseCursor so the pointer (top left had corner) appears to be touching the
item. When you point to something, a small box often appears that describes the
item.

2)      The red circle component – This
component is used to display where the tip of user’s nose is located.

3)      The black cross-hair component – This
component works with the red circle to assist the user in understanding how to navigate
NouseCursor on the screen. Refer to the next section for a detailed description
of how the navigation works. The center of the cross hair is a neutral or stop
position.

4)      The text component at the bottom of NouseCursor
– This component provides feedback to the user with respect to which action or
mode is active for NouseCursor. The feedback is provided using words (e.g.
“joystick”) and colour changes (e.g. blue to yellow). The details of each
action/mode are described throughout this user guide.

5)      The user’s face background component
(assuming this option is active) – In the background of NouseCursor is the
video feed of the user’s face. The user can see their face and more
specifically where their nose is located. The user can also utilize a standard
background as in Figure 3.1.

6)      The standard Microsoft Windows cursor is
embedded in the NouseCursor. See top left corner of the figure below.

3.2 Nouse Navigation Modes

Nouse application provides the users with two modes for navigating
the computer mouse namely in “Joystick” or “Direct” mode. The mode the user
should select is dependent on the user’s preferences. There are pros and cons
to each mode and the user should select the mode that is most comfortable for
them. By default Nouse application comes in “Joystick” mode.

3.2.1 Joystick Navigation Mode (Default Mode)

With joystick mode, the movement of NouseCursor is determined
by the position of the nose relative to the center of the black cross-hair
component of NouseCursor (see black cross-hair defined in figure 3.1). The
location of the nose is represented by the red circle component of NouseCursor.
When the user’s nose (or red circle) is in the center of the cross-hair, NouseCursor
remains still. If the user moves their nose slightly away from the center of
the cross-hair in any direction, NouseCursor will move slowly in that direction
across the screen and will continue to move until the user brings their nose
back to the center of the cross-hair. The farther away the nose is from the
center of the cross-hair, the faster NouseCursor will move across the screen. This
mode is typically easier to use and easier to select more precise points on the
screen than “Direct” mode. Figure 3.1 shows NouseCursor in joystick navigation
mode.

3.2.2 Direct Navigation Mode

With the Direct navigation mode, the movement of NouseCursor
is mapped directly to the user’s nose movements. In this mode the user will
feel as if NouseCursor is directly attached to their nose. If the user’s nose moves
to the far left of their range of motion, NouseCursor will immediately go to
the far left of the screen. The same will be true for the movement to any
location on the screen. In “Direct” mode the user can typically move NouseCursor
faster than in “Joystick” mode. However, in “Direct” mode it is more difficult
to select a very precise location on the screen as any movement of the user’s
nose will cause a movement of NouseCursor. Only users who have very good
control of their head movements should utilize this mode. Figure 3.2 shows NouseCursor
in direct navigation mode.

Figure 3.2:
NouseCursor using Direct Navigation mode

Hint: In order to
change from Direct to Joystick or from Joystick to Direct, the users will need
to make the change in the settings program and restart Nouse application (see Nouse
Setting Guide).

3.3 Nouse Clicking Mode

Nouse application allows the user to reproduce the “clicks”
of a standard computer mouse. The standard mouse has “Left Click”, “Right
Click”, “Drag” and “Double Click”. These can be reproduced hands-free using Nouse
application in a very unique manner.

As mentioned above, the top left hand corner of NouseCursor
represents the pointer to where the user would like to click. When the user
needs to utilize a mouse “click”, NouseCursor is moved to the desired clicking
location. The user stops NouseCursor at the clicking point. Once NouseCursor is
still for 1 to 2 seconds, the user will see a countdown begin. The default
countdown starts at 3 and counts down to 0.

If the user does not move NouseCursor during the countdown
then when the counter hits zero (0) NouseCursor moves to the first level clicking
mode. Upon entering clicking mode, the text component of NouseCursor becomes
green and displays the word "Click!” Figure 3.3 shows NouseCursor in click
mode.

If the user moves NouseCursor before the countdown reaches
zero (0) then the countdown is reset and NouseCursor is allowed to continue to
move in navigation mode.

Figure 3.3: Move to
select click type or stay still to do nothing

Once the user is in clicking mode the user has one of four clicking
choices:

1)      Move
the Nose left
in order to perform a left click.

2)      Move
the Nose right
in order to perform a right click.

3)      Move
the Nose down
in order to perform a double click.

4)      Move
the Nose up
in order to bring up a submenu of other actions. Figure 3.4 shows NouseCursor
with the submenu of selectable actions.

Figure 3.4: The submenu
options on a Nose “up” selection of click mode

The user can select the desired action from the submenu by
moving NouseCursor up or down along the menu options, holding for the countdown
and then selecting the left click.

The submenu consists of 7 options for the user to select.
The Drag option is described in detail below. The remaining options (i.e.,
Pause, NouseBoard, Recalibrate, Settings, Disable, and Exit) are shortcuts for
the buttons found on Nouse Interface window. These are all described in section
3.4.

Drag Option

When the user selects the Drag submenu option, a letter “D”
is drawn in the top left corner of NouseCursor and the user is in standard
mouse drag mode. In order to stop dragging, the user must click again in the
desired stop location. Figure 3.5 shows NouseCursor in dragging mode.

Figure 3.5: The ’D’
means that a drag is being executed

When the user clicks in the stop location the area from the
point where the user selected the drag menu option to the stop location is
highlighted (i.e., this is the same as a standard mouse drag selection). The
user can then determine what to do with the selected area. In order to
de-select the area the user can click anywhere and the selected area becomes
unhighlighted.

HINT – Nouse
application provides the user with several other methods for speeding up the
ability to execute mouse clicks. This includes using “Voice Commands” described
in section 3.6 of this guide or “Use Mouth Clicks” or “Enable Dwell Clicking”
detailed in the Nouse Settings Guide.

3.4 Nouse Interface

Nouse interface (shown in Figure 3.6) is activated when Nouse
application is started. The interface allows the user access to change key
Nouse functionality and settings. By default this window is minimized when Nouse
application starts. The user can also access Nouse functionality and setting
from the submenu described in section 3.3.

HINT – This small
window can be minimized when not needed.

The following section will describe the different options in
Nouse interface.

3.4.1 Camera Feed

In the center of Nouse interface is a rectangle showing the
video being captured by the webcam. This window is provided so that the user
can confirm the Webcam is working and that the user’s face is visible within
the webcam viewing range.

Figure 3.6: The main
Nouse interface

3.4.2 Pause Button

At times the user will need to temporarily stop the actions
of NouseCursor from being active. This will include times where the user is
watching a movie or reading a document. In order to inactivate or pause NouseCursor,
the user can select pause mode by pressing the "Pause" button. In
pause mode NouseCursor disappears from the screen and only the standard mouse
pointer can be seen. The user can continue to navigate, however all clicking,
countdown and typing functionality is turned off.

To exit the “Pause” mode the user simply moves the standard
cursor to the bottom right corner of the screen. NouseCursor will re-appear and
the user is placed in navigation mode.

3.4.3 NouseBoard Button

Nouse application provides the user with the ability to
enter text without the need to use the standard keyboard. The “NouseBoard” is a
keyboard that works with Nouse application to allow a user to type hands-free. Clicking
on the “NouseBoard” button will activate Nouse keyboard. A detailed description
of NouseBoard is provided in section 3.5 of this guide.

3.4.4 Recalibrate Button

Clicking on the "Recalibrate" button brings the
user to Calibration mode (see section 2.5). The
recalibration of NouseCursor allows the user to reset the way the nose is being
tracked. This only needs to be executed when the user’s nose is not being
tracked properly by Nouse application.

3.4.5 Settings Button

Nouse application allows the user to modify the
configuration and features of the program to suit personal preferences. These
modifications can then be saved and stored so that the next time the user
starts Nouse application it uses the preferences saved by the user. There are
many options that can be modified in Nouse application. All of Nouse setting options
are discussed in detail in the Settings Guide. To change the settings, the user
clicks on the "Settings" button which brings up Nouse Settings
program.

3.4.6 Disable/Enable Button

Clicking on the "Disable/Enable" button will stop
and re-start Nouse activities. To Disable the application the user clicks on
the “Disable” button. The application will then ask the user to confirm they
would like to Disable. If the user selects “Yes” button then the application is
disabled. When Nouse is disabled, Nouse Interface window remains open but there
is no NouseCursor. To “Enable” the user can activate Nouse Interface window and
select the “Enable” button.

HINT: It is
important to note that if the user disables the application then they will not
be able to utilize their computer hands-free. Once Nouse application is
disabled the user must use their standard mouse to enable Nouse application.

3.4.7 Exit Button

The user can exit the application when they are finished. To
exit the application the user clicks on the "Exit" button. The
application will then ask the user to confirm they would like to exit. If the
user selects the “Yes” button then the application is closed. If the user
select the “No” button then the user is returned to Nouse Interface screen.

HINT: It is
important to note that if the user exits the application then they will not be
able to utilize their computer hands-free. Once Nouse application is closed the
user must use their standard mouse to perform any further actions.

HINT – Nouse
application provides the user with another option to execute the functions of Nouse
interface by using Voice commands. This is described in detail in section 3.6.

3.5 NouseBoard Mode

Nouse application provides the user with the ability to
enter text without the need to use the standard keyboard. The “NouseBoard” is
an on-screen configurable keyboard that works with Nouse application to allow a
user to type hands-free. To activate NouseBoard the user can click on the
“NouseBoard” button from Nouse Interface or from NouseCursor submenu. NouseBoard
will be displayed as shown in Figure 3.7.

Figure 3.7: Standard
NouseBoard and NousePad

3.5.1 Using NouseBoard

NouseBoard is configured to work in direct navigation mode
only. That is, the movement of NouseCursor is mapped directly to the user’s
nose movements. In this mode the user will feel as if NouseCursor is directly
attached to their nose. If the user’s nose moves to the far left of their range
of motion, NouseCursor will immediately go to the far left of the screen. The
same will be true for the movement to any location on the screen.

NouseBoard keyboard consists of a grid of 15 cells, 5
columns by 3 rows. Each cell consists of 4 letters/numbers/symbols or actions.
When in NouseBoard mode NouseCursor moves from cell to cell in direct mode. In
order to display the text that is typed, NouseBoard uses a second window located
below NouseBoard to display and edit the text. This second window is called NousePad.
NousePad is described in detail below.

Note: Once NouseBoard
is activated the user remains in NouseBoard window until the “Quit” is
selected.

3.5.2 Cell Selection

In order to select or type a letter/number or action the
user must first move NouseCursor over the cell that has the desired letter/number
or action. To do this the user simply needs to move their nose in the direction
of the desired cell to activate that cell. Unlike in “Joystick” mode, the user
must continue to point their nose in the direction of the selected cell in
order to keep it selected.

For example, if the user’s nose is in the center of the
cross-hair, the center cell of NouseBoard will be selected. If the user moves their
nose up, the top center cell will be selected. If the user moves their nose up
and a bit to the left, the second cell in the top row will be selected. Figure
3.8 shows the cell located in column 3 row 2 selected (i.e., purple or bolded
highlight around the cell)

Figure 3.8: Selecting
a Cell with NouseBoard

Each cell in the grid contains four typing options. The
first step in typing a letter is to select the cell that contains the letter. To
select a letter the user moves the highlight to the desired cell in NouseBoard and
remains still. A countdown will begin much like when clicking in NouseCursor
Mode. If the user moves, the countdown will restart. Once the countdown reaches
zero, the user is moved into select mode (or select an option).

3.5.3 Letter/Option Selection

Just like with “Nouse clicking” the user has five options:
move in one of four directions or do nothing. Once the desired cell is selected
and the countdown has reached zero, the user can select the desired letter by moving
NouseCursor in the corresponding direction. For example, if the user has
selected the cell that contains the letters “f”, “e”, “h” and “g”, the user can
type ’f’ by moving NouseCursor left, ’e’ by moving up, ’h’ by moving right or
’g’ by moving down. If the user does nothing and the countdown reaches zero,
Nouse goes back to cell selection mode. Figure 3.9 provides an example of the
sentence “nouse is fun!” having been typed using NouseBoard.

Figure 3.9: Using
NouseBoard with NousePad

3.5.4 NousePad Text Paste

NouseBoard cannot type directly into a third party
application such as Microsoft Word or a Web browser. Instead, Nouse application
uses NousePad to display and capture the text entered by the user. In order to
move or type the text into another application, the user simple edits the text
in NousePad using NouseBoard, exits from NouseBoard, goes to the application
and selects the paste action of that application. The text created in NousePad will
then be pasted into the application.

Another method to perform the paste action is to utilize the
first left click or double click after exiting NouseBoard. Upon exiting
NouseBoard, NouseCursor will display a "*" character (see Figure 3.10)
in the top left hand corner if there was something copied to the clipboard.
This "*" indicates there is text to "paste".

Figure 3.10: The star
indicates there is something to paste

HINT: All text
that is entered into NousePad is automatically copied to the Windows clipboard
upon exit from NouseBoard. It is then available to be pasted into any
applications that support the paste action.

3.5.5 Editing Text

When a user types a character in NouseBoard mode the
character is displayed in NousePad window found just below NouseBoard. The user
continues to type until all the text is completed.

If the user makes a mistake typing a letter or would like to
change the text then the user can select one of NouseBoard text edit actions for
editing text. These actions mimic the standard keyboard actions for editing
text. Below is a summary of a few important actions:

·
Shift: Found in column 2 row 3 of
NouseBoard (when the Shift is not already pressed). This action is the same as
the Shift key of a standard keyboard (e.g., capitalizing letters). Selecting
this action will produce a second keyboard layout (see Figure 3.11). The second
keyboard layout has all the capital letters.

·
BKSP: Found in column 3 row 3of
NouseBoard. This action is the same as the BackSpace key of a standard keyboard
(i.e., deletes the previously typed character).

·
Quit: Found in column 3 row 3of
NouseBoard. This exits NouseBoard and goes back to NouseCursor mode.

·
Pause (for NouseBoard): This
action puts NouseCursor in Pause mode (i.e., no countdowns or typing). In pause
mode, you can move the pink square around the board but there is no typing
functionality or countdowns. To exit pause mode, navigate to the bottom right
cell of NouseBoard.

1                             2                            3                              4                           5

1

2

3

Figure 3.11: Standard
NouseBoard and NousePad with the Shift key selected

NouseBoard has many more options including word prediction
and the ability to change the layout of the keyboard. For a complete list of
NouseBoard options, see Nouse Settings Guide.

3.6 Voice Commands

Nouse application provides a method for executing commands
and mouse clicks using the user’s voice. Users can say commands and have them
executed automatically. When properly configured, voice commands can make Nouse
activities faster and easier to use.

Nouse application is setup to execute (via voice) a
predetermined set of commands. For example, a voice command can be used to
execute a right click or to disable Nouse. The user simply says “Right” for a
right click or “Disable” to disable Nouse application. In order to utilize the
voice commands with Nouse, the user must first set up a third party voice
recognition application.

Nouse application is capable of working with general speech
recognition applications. Users must follow the installation and setup
instructions that come with the speech recognition software before using it
with Nouse.

Microsoft Windows provides a Speech recognition application
by default. To activate this application the user can go to the Control Panel
of Microsoft Windows Start button. See Figure 3.12a and b for the different
operating systems.

Figure 3.12a –
Windows XP or earlier: Click on Control Panel -> Speech

Figure 3.12b -
Windows 7 Users: Click on Control Panel -> Ease of Access -> Speech
Recognition

The user must follow the instruction to properly setup and
configure the Speech Recognition application. Once the application has been installed,
the user must “Train” the application to recognize the user’s voice commands.
See setup instructions and videos that come with the speech application.

Once the user has trained the speech recognition application
to recognize the user’s voice, the user must enable the voice commands by
setting the "Use Voice Commands" option (see Nouse Settings Guide) in
Nouse application. Note – The user does not need the Speech Recognition
application to be active in order to use the voice commands with Nouse.

This section will describe the various voice commands
available to a Nouse User. When the user is in Navigation mode the following
commands can be executed. To execute the command the user says the word in
bold:

Clicks Commands

·
Click – This will move the user from
navigation mode to click mode. The user can then select the desired click by
moving their nose in the standard manner. Note: If the user enables the “Click
on Command” (see Nouse Settings Guide) then the Click voice command will automatically execute a left click.

·
Up –
This will bring up the submenu (see section 3.3). The user can then select the
desired submenu option by moving their nose in the standard manner.

·
Right – Performs a right click.

·
Double – Perform a double click.

Other Commands

·
Disable - Turn off Nouse application.

·
Enable - Turn Nouse back on if it has
been disabled.

·
Settings – Brings up Nouse Setting
configuration window (see Nouse Settings Guide).

·
Pause - Pause Nouse application or unpause
if Nouse application has been paused.

·
Board – Start NouseBoard, or exit
NouseBoard if it is currently on. See other voice commands below for the
NouseBoard.

When
the user is in NouseBoard mode the following commands can be executed. To
execute the command the user says the word in bold:

·
Up – This will type the letter (or select
the option) at the top of the currently selected cell.

·
Down – This will type the letter (or
select the option) at the bottom of the currently selected cell.

·
Left – This will type the letter (or
select the option) on the left of the currently selected cell.

·
Right – This will type the letter (or
select the option) on the right of the currently selected cell.