Kinect for windows sdk windows 10 x64

• [Kinect for Windows SDK. Часть 1. Сенсор]
• Kinect for Windows SDK. Часть 2. Потоки данных
• Kinect for Windows SDK. Часть 3. Функциональные возможности
• Играем в кубики с Kinect
• Программа, апорт!

Три части обзорной статьи можно считать вводными к двум другим: «Играем в кубики с Kinect» и «Программа, апорт!». Хотя хронологически они идут последними. Более того, за время их написания Microsoft успел выпустить новую версию SDK – 1.5. Какой удар со стороны корпорации!

Как вам должно быть известно, Kinect – это бесконтактный контроллер, т.е. скажем в играх, вместо того, чтобы яростно стучать по клавиатуре или терзать gamepad, вы размахиваете руками и ногами и время от времени голосом даете какие-то команды. Самое время спрятать дорогие китайские вазы X века подальше!

Начинать знакомство с Kinect лучше со знакомства с самим сенсором. Это небольшая черная коробочка с тремя камерами, длиной чуть более 20 см и высотой около 5 см (не считая ножки). Один нюанс – в Kinect две камеры. И это изображение (взято из MSDN) тому подтверждение.

• IR Emitter – Инфракрасный излучатель. Да, это не камера. Его назначение испускать инфракрасные лучи, которые, отражаясь от предметов, попадают назад в сенсор, где их принимает:
• IR Depth Sensor – Инфракрасный приёмник. Собирает отраженные лучи, преобразуя их в расстояние от сенсора до объекта(ов). И таким образом строится, можно сказать, матрица расстояний — целый кадр. Максимальное разрешение 640×480 (30 fps). Если говорить о переводе мне кажется, «ИК приёмник» лучше отражает суть, чем дословный «ИК датчик глубины».
• Color Sensor – Цветная камера. Захват видео с максимальным разрешением 1280×960 (12fps). Угол обзора камеры: 43° по вертикали и 57° по горизонтали (угол обзора дальномера идентичный). Можно выбрать формат картинки: RGB или YUV.
• Microphone Array – набор микрофонов. 4 встроенных микрофона позволяют определить местоположение источника звука и направление звуковых волн. Встроенный обработчик звукового сигнала включает подавление эха и уменьшение шума.
• Tilt Motor – Коррекция наклона. Позволяет программно настраивать наклон сенсора в диапазоне ±27° по вертикали.

Сенсор подключается к компьютеру через USB, хотя, строго говоря, USB устройством не является, т.к. имеет несколько отличную прошивку, нежели стандартное USB устройство. Найти какие-либо комментарии по этому поводу не получилось, но можно предположить, что сделано это исключительно в целях повышения производительности. Вы можете подключить до четырех сенсоров одновременно.

Другой важный аспект аппаратной части – диапазон «рабочих» расстояний, т.е. на каком расстоянии сенсор способен видеть объекты (данные дальномера). И снова картинка помогает продемонстрировать возможности Kinect.

Режим Default Range доступен как для Kinect for Windows, так и для Kinect for XBox, но Near Range доступен только для сенсора Kinect for Windows. Простой пример: для всех объектов в кадре находящихся в пределах Normal Values сенсор вернёт измеренное расстояние, но для объектов, выходящих за этот диапазон, сенсор вернёт либо ноль, либо определенную для каждого диапазона константу (зависит от заданных вами настроек). Одно из существенных отличий SDK 1.5 от 1.0 в том, что теперь в режиме Near Range можно получить информацию о сидящем человеке, это так называемый трекинг сидящей фигуры (seated skeletal tracking). Kinect отслеживает 10 точек: голову, шею и руки. Но об этом чуть позже.

Хочется обратить внимание на различия двух Kinect (XBox и Windows). Первый разрабатывался и тестировался с одним устройством – XBox. Второй же, напротив, должен работать с широким диапазоном устройств (компьютеров), поэтому его кабель короче, а специальное устройство призвано не позволить Kinect подраться с остальной USB периферией. Лицензионное соглашение позволяет использовать Kinect for Windows в коммерческих целях, а Kinect for XBox нет.

Перейдем теперь к установке SDK. В версии 1.5 Microsoft разделил SDK на два пакета. Скачать их можно с официальной страницы:

• Kinect for Windows SDK. Включает APIs и драйверы сенсора.
• Kinect for Windows Developer Toolkit. Включает Kinect Studio (полезный для отладки инструмент), Face Tracking SDK, примеры кода и документация.

Не будем останавливаться на установке SDK, в этом нет ничего сложного или интересного. А вот что действительно интересно так это то, что же разработчик может получить от Kinect? Взглянем на следующие возможности:

• Skeletal Tracking (трекинг человеческой фигуры)
• Face Tracking (трекинг лица)
• Speech Recognition (распознавание речи)

Обзор этих функций вы найдете в следующих частях статьи.

Полезные ссылки:

• Kinect for Windows SDK
• Руководство разработчика. Можно найти примеры на C# (managed code) и на C/C++ (unmanaged code)
• Kinect for Windows Quickstart Series. Серия видео-уроков о возможностях Kinect (весьма полезная для знакомства с устройством).
• Near Mode: What it is (and isn’t). Статья о near mode, хотя написана в преддверии SDK 1.0, но полезна в плане разоблачения некоторых мифов.
• Форумы Kinect for Windows SDK. Здесь можно обратиться за советом к экспертам.

Microsoft Kinect for Windows SDK v2. Provides data source APIs for Kinect.

This package has no dependencies.

This version works with Kinect for Windows v2.0. We recommend you install the SDK for access to samples and tools.

This release introduces support for the Kinect for Windows v2 sensor, and introduces a broad range of capabilities for developers. The Kinect for Windows SDK 2.0 includes the following:

• Drivers for using Kinect v2 sensors on a computer running Windows 8 (x64), Windows 8.1 (x64), and Windows Embedded Standard 8 (x64)
• Application programming interfaces (APIs) and device interfaces
• Code samples

To install the Kinect for Windows SDK 2.0:

1. Make sure the Kinect sensor is not plugged into any of the USB ports on the computer.
2. From the download location, double-click on KinectSDK-v2.0_1409-Setup.exe
3. Once the Kinect for Windows SDK has completed installing successfully, ensure the Kinect sensor is connected to the power hub and the power hub is plugged into an outlet. Plug the USB cable from the power hub into a USB 3.0 port on your computer. Driver installation will begin automatically.
4. Wait for driver installation to complete. You can verify that installation has completed by launching Device Manager and verifying that «KinectSensor Device» exists in the device list. Note:On first plugin, the firmware on the device will be updated. This may result in the device enumeration happening several times in the first minute.
5. Installation is now complete.

What’s New:

Windows Store Support

With this release of Kinect for Windows, you are able to develop and publish Kinect enabled applications which target the Windows Store. We are incredibly excited to see what people create. All of the Kinect SDK and sensor functionality are available in this API surface, except for Speech. For more information about developing Windows Store applications which use Kinect, please see: https://go.microsoft.com/fwlink/?LinkId=517592.

Unity Support

For the first time, the Kinect API set is available in Unity Pro, through a Unity Package. We are excited to be able to offer the platform to our developers. APIs for Kinect for Windows core functionality, visual gesture builder and face are now available to be called from Unity apps. The Unity plugins are available for download at: https://go.microsoft.com/fwlink/?LinkID=513177

.NET APIs

The Managed API set should feel familiar to developers who worked with our managed APIs in the past. We know this is one of the fastest development environments available, and that many development shops have an existing investment in this space. All of the Kinect SDK and sensor functionality are available in this API surface.

Native APIs

Many Kinect applications require the full power and speed that writing native C++ code requires. We are excited to share this iteration of the native APIs for Kinect. The form and structure of the APIs is identical to the Managed API set, but allow a developer to access the full speed of C++. These APIs are a significant divergence from the v1.x native APIs, and should be significantly easier to use. All of the Kinect SDK and sensor functionality are available in this API surface.

Audio

The Kinect sensor and SDK provide a best in class array microphone and advanced signal processing to create a virtual, software based microphone which is highly directional, and which can understand the direction sounds are coming from. In addition, this provides a very high quality input for Speech recognition.

Face APIs

Extended massively from v1, the Face APIs provide a wide variety of functionality to enable rich face experiences and scenarios. Within the Face APIs, developers will be able to detect faces in view of the sensor, align them to 5 unique facial identifiers, and track orientation in real-time. With HD Face, the technology provides 94 unique «shape units» per face, to create meshes with a very rich likeness to the user. The meshes can be tracked in real-time to show rich expressiveness and live tracking of the user’s facial movements and expressions.

Kinect for Windows v2 Hand Pointer Gestures Support

If you would like to enable your applications to be controls through hand pointer gestures, Kinect for Windows v2 has improved support. See ControlsBasics-XAML, ControlsBasics-WPF and ControlsBasics-DX for examples of how to hand pointer gesture enable your applications. This is an evolution of the KinectRegion/KinectUserViewer support that we provided in Kinect for Windows v1.7 and later. KinectRegion and KinectUserViewer are available for XAML and WPF applications. The DirectX support is built on top of a lower level Toolkit Input component.

Kinect Fusion

With this release of Kinect for Windows, you are able develop and deploy Kinect Fusion applications. This provides higher resolution, better camera tracking and performance than the 1.x releases of Kinect Fusion.

Kinect Studio

Kinect Studio has had a major rewrite since the v1 days, in order to handle the new sensor, and to provide users with more customization and control. The new user-interface offers flexibility in the layout of various workspaces and customization of the different views. It is now possible e.g. to compare two 2D or 3D views side-by-side or to create a custom layout to meet your needs. The separation of the monitoring, recording and playback streams exposes additional functionality such as file- and stream-level metadata. The timeline features: in- and out-points to control what portion of the playback to play; pause-points that let you set multiple points at which to suspend a playback; markers, that let you attach meta-data to various points in time. This preview also exposes playback looping and additional 2D/3D visualization settings. There is still some ‘placeholder’ artwork here and there, but the tooltips should guide you along.

Visual Gesture Builder (Preview)

Introducing Visual Gesture Builder, a gesture detector builder that uses machine-learning and body-frame data to ‘define’ a gesture. Multiple body-data clips are marked (aka ‘tagged’) with metadata about the gesture which is then used by a machine-learning trainer during the build step to extract a gesture definition from the body-data clips. The gesture definition can subsequently be used by the gesture detection runtime — called by your application — to detect one or more gestures. While using machine-learning for gesture detection is not for the faint of heart, it offers a path to rapid prototyping. Using vgbview, you can benchmark your gesture definitions without requiring that you write any code. For detailed walkthrough videos and a whitepaper on using VGB, please see the resources at: https://social.msdn.microsoft.com/Forums/en-US/02e0302a-e3bd-46d3-9146-0dacd11d2a8d/deep-dive-videos-and-whitepaper-for-visual-gesture-builder?forum=kinectv2sdk.

(For more resources related to this topic, see here.)

System requirements for the Kinect for Windows SDK

While developing applications for any device using an SDK, compatibility plays a pivotal role. It is really important that your development environment must fulfill the following set of requirements before starting to work with the Kinect for Windows SDK.

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com.
If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support. and register to have the files e-mailed directly to you.

Supported operating systems

The Kinect for Windows SDK, as its name suggests, runs only on the Windows operating system. The following are the supported operating systems for development:

• Windows 7

• Windows Embedded 7

• Windows 8

The Kinect for Windows sensor will also work on Windows operating systems running in a virtual machine such as Microsoft HyperV, VMWare, and Parallels.

System configuration

The hardware requirements are not as stringent as the software requirements. It can be run on most of the hardware available in the market. The following are the minimum configurations required for development with Kinect for Windows:

• A 32- (x86) or 64-bit (x64) processor

• Dual core 2.66 GHz or faster processor

• Dedicated USB 2.0 bus

• 2 GB RAM

The Kinect sensor

It goes without saying, you need a Kinect sensor for your development. You can use the Kinect for Windows or the Kinect for Xbox sensor for your development.

Before choosing a sensor for your development, make sure you are clear about the limitations of the Kinect for Xbox sensor over the Kinect for Windows sensor, in terms of features, API supports, and licensing mechanisms.

The Kinect for Windows sensor

By now, you are already familiar with the Kinect for Windows sensor and its different components. The Kinect for Windows sensor comes with an external power supply, which supplies the additional power, and a USB adapter to connect with the system. For the latest updates and availability of the Kinect for Windows sensor, you can refer to http://www.microsoft.com/en-us/kinectforwindows/site.

The Kinect for Xbox sensor

If you already have a Kinect sensor with your Xbox gaming console, you may use it for development. Similar to the Kinect for Windows sensor, you will require a separate power supply for the device so that it can power up the motor, camera, IR sensor, and so on.

If you have bought a Kinect sensor with an Xbox as a bundle, you will need to buy the adapter / power supply separately. You can check out the external power supply adapter at http://www.microsoftstore.com. If you have bought only the Kinect for Xbox sensor, you will have everything that is required for a connection with a PC and external power cable.

Development tools and software

The following are the software that are required for development with Kinect SDK:

• Microsoft Visual Studio 2010 Express or higher editions of Visual Studio

• Microsoft .NET Framework 4.0 or higher

• Kinect for Windows SDK

Kinect for Windows SDK uses the underlying speech capability of a Windows operating system to interact with the Kinect audio system. This will require Microsoft Speech Platform – Server Runtime, the Microsoft Speech Platform SDK, and a language pack to be installed in the system, and these will be installed along with the Kinect for Windows SDK. The system requirements for SDK may change with upcoming releases. Refer to http://www.microsoft.com/en-us/ kinectforwindows/. for the latest system requirements.

Evaluation of the Kinect for Windows SDK

Though the Kinect for Xbox sensor has been in the market for quite some time, Kinect for Windows SDK is still fairly new in the developer paradigm, and it’s evolving. The book is written on Kinect for Windows SDK v1.6. The Kinect for Windows SDK was first launched as a Beta 1 version in June 2011, and after a thunderous response from the developer community, the updated version of Kinect for Windows SDK Beta 2 version was launched in November 2011. Initially, both the SDK versions were a non-commercial release and were meant only for hobbyists. The first commercial version of Kinect for Windows SDK (v1.0) was launched in February 2012 along with a separate commercial hardware device. SDK v1.5 was released on May 2012 with bunches of new features, and the current version of Kinect for Windows SDK (v1.6) was launched in October 2012. The hardware hasn’t changed since its first release. It was initially limited to only 12 countries across the globe. Now the new Kinect for Windows sensor is available in more than 40 countries. The current version of SDK also has the support of speech recognition for multiple languages.

Downloading the SDK and the Developer Toolkit

The Kinect SDK and the Developer Toolkit are available for free and can be downloaded from http://www.microsoft.com/en-us/kinectforwindows/.

The installer will automatically install the 64- or 32-bit version of SDK depending on your operating system. The Kinect for Windows Developer Toolkit is an additional installer that includes samples, tools, and other development extensions. The following diagram shows these components:

The main reason behind keeping SDK and Developer Toolkit in two different installers is to update the Developer Toolkit independently from the SDK. This will help to keep the toolkit and samples updated and distributed to the community without changing or updating the actual SDK version. The version of Kinect for Windows SDK and that for the Kinect for Windows Developer Toolkit might not be the same.

Installing Kinect for Windows SDK

Before running the installation, make sure of the following:

• You have uninstalled all the previous versions of Kinect for Windows SDK

• The Kinect sensor is not plugged into the USB port on the computer

• There are no Visual Studio instances currently running

Start the installer, which will display the start screen as End User License Agreement. You need to read and accept this agreement to proceed with the installation. The following screenshot shows the license agreement:

Accept the agreement by selecting the checkbox and clicking on the Install option, which will do the rest of the job automatically.

Before the installation, your computer may pop out the User Access Control (UAC) dialog, to get a confirmation from you that you are authorizing the installer to make changes in your computer.

Once the installation is over, you will be notified along with an option for installing the Developer Toolkit, as shown in the next screenshot:

Is it mandatory to uninstall the previous version of SDK before we install the new one?

The upgrade will happen without any hassles if your current version is a non-Beta version. As a standard procedure, it is always recommended to uninstall the older SDK prior to installing the newer one, if your current version is a Beta version.

Installing the Developer Toolkit

If you didn’t downloaded the Developer Toolkit installer earlier, you can click on the Download the Developer Toolkit option of the SDK setup wizard (refer to the previous screenshot); this will first download and then install the Developer Toolkit setup. If you have already downloaded the setup, you can close the current window and execute the standalone Toolkit installer. The installation process for Developer Toolkit is similar to the process for the SDK installer.

Components installed by the SDK and the Developer Toolkit

The Kinect for Windows SDK and Kinect for Windows Developer Toolkit install the drivers, assemblies, samples, and the documentation. To check which components are installed, you can navigate to the Install and Uninstall Programs section of Control Panel and search for Kinect. The following screenshot shows the list of components that are installed with the SDK and Toolkit installer:

The default location for the SDK and Toolkit installation is %ProgramFiles%/Microsoft SDKs/Kinect.

Kinect management service

The Kinect for Windows SDK also installs Kinect Management, which is a Windows service that runs in the background while your PC communicates with the device. This service is responsible for the following tasks:

• Listening to the Kinect device for any status changes

• Interacting with the COM Server for any native support

• Managing the Kinect audio components by interacting with Windows audio drivers

You can view this service by launching Services by navigating to Control Panel |Administrative Tools, or by typing Services.msc in the Run command.

Is it necessary to install the Kinect SDK to end users’ systems?

The answer is No. When you install the Kinect for Windows SDK, it creates a Redist directory containing an installer that is designed to be deployed with Kinect applications, which install the runtime and drivers. This is the path where you can find the setup file after the SDK is installed:

%ProgramFiles%/Microsoft SDKsKinectv1.6Redist KinectRuntime-v1.6-Setup.exe

This can be used with your application deployment package, which will install only the runtime and necessary drivers.