Give other hand some control

Yepher · February 23, 2021, 4:18pm

I really like drawing in Shapr3D but I find the fingers, on one hand, are mostly useless during most workflows. I use my thumb for some screen functions but the other digits are just for holding the iPad.

So I am working on a little macro keyboard (from some spare parts) that will attach to the back of my iPad with eight buttons (2 per finger). I will map it to the primary views top/bottom,default and Select All and De-Select all. May try a little joystick to see if that is better than buttons.

Here is a demo of a two-button breadboard proto-type I am playing with

TigerMike · February 23, 2021, 5:05pm

Great idea Chris!

If possible, I’d be delighted to beta test
My only request is that it be compatible for us Lefties as I for one swap the screen tools left to right.

-Mike

milouchien · February 23, 2021, 5:31pm

Quite brilliant!

Oregonerd · February 23, 2021, 6:42pm

Details man, Details! I assume it’s BT connected, Which dev board is that? If you don’t mind divulging.

Yepher · February 23, 2021, 7:25pm

Yes, it is a BT HID. I am using a random ESP32 that I had laying around. This particular one is the Heltec HTIT-WB32. That board is overkill for what I need but it has a built-in charging circuit so it makes for easy prototyping stuff, and I have used it a lot for other projects. If I get really crazy I may make a little companion iPad app to make custom configuration easier.

At the moment I plan to put all the plans to build it in GitHub once it is a little further along. Still sorting out a better form factor before I design a board and case for it.

One problem I am not sure how to solve, yet, is the “thing” to connect it to an iPad so it can fit most iPads… And how to make it usable for lefties and righties. I sort of have an idea at least for the ambidextrous part.

Robert · February 23, 2021, 7:27pm

I will definitely buy one

Steve · February 23, 2021, 7:28pm

Wow, phenomenally good idea. I assume you are emulating a BT keyboard with the dev board. Which means that, thanks to the new search function, you could pretty much map any button to any desired function.

Oregonerd · February 23, 2021, 7:38pm

Can you do capsense with it?

Steve · February 23, 2021, 7:40pm

You might consider attaching it with magnets. That is how the Smart Keyboard is attached and there are A LOT of small plus a few larger magnets embedded in the back of the iPad. Easy to locate with a cheap magnetic viewing film.

Built a handy iPad stand for my shop. Quick release and attach, plus surprisingly strong. I do suggest also using a very thin, high friction film such as rubber to prevent lateral movement.

TigerMike · February 23, 2021, 7:42pm

One simple way to connect to an iPad is with velcro. That way it connect directly to the back of an iPad or to an iPad case, or the Magic Keyboard as what I have. I too would purchase one.
-Mike

Yepher · February 23, 2021, 7:41pm

I have a few times before. But since my plan is for the buttons to be behind the iPad I think it may need to be physical buttons that are not easy to accidentally trigger…

Oregonerd · February 23, 2021, 7:51pm

There is one BT module that you repurpose the unused GPIO with At commands. Had to be from a certain company though. HC05 module, super cheap. No need for separate MCU. I was playing with that in App inventor. (My programming skills are a bit weak). Lost the info on HD crash.

symonty · February 24, 2021, 4:45am

Why not use a digispark and just plug it into the ipad , it will give you power and you can easily create a fewbuttons with a simple bit of code. ( and no shift register ) . BT seems overkill since you still need power.
Something like this but less complex.

Oregonerd · February 24, 2021, 5:29am

I hate wires!

symonty · February 24, 2021, 6:31am

Good luck then, the ESP32 BT stack aint that grand and I see reports of issues connecting to Ipad Devices with even simple keyboard emulation.

I hate wires too, but cables are more reliable.

PS: I am a big proponent of the ESP32 and ESP8266 have at least 40 projects complete on that platform, multiples using ATTiny too, so I see the draw for using the esp32 for sure. Just had no much luck with BT stack on ios devices beyond beconing.

gist.github.com

https://gist.github.com/manuelbl/66f059effc8a7be148adb1f104666467

README.md

# ESP32 as Bluetooth Keyboard

With its built-in Bluetooth capabilities, the ESP32 can act as a Bluetooth keyboard. The below code is a minimal example of how to achieve it. It will generate the key strokes for a message whenever a button attached to the ESP32 is pressed.

For the example setup, a momentary button should be connected to pin 2 and to ground. Pin 2 will be configured as an input with pull-up.

In order to receive the message, add the ESP32 as a Bluetooth keyboard of your computer or mobile phone:

1. Go to your computers/phones settings
2. Ensure Bluetooth is turned on

This file has been truncated. show original

main.cpp

/* 
 * Sample program for ESP32 acting as a Bluetooth keyboard
 * 
 * Copyright (c) 2019 Manuel Bl
 * 
 * Licensed under MIT License
 * https://opensource.org/licenses/MIT
 */

//

This file has been truncated. show original

Yepher · February 24, 2021, 1:50pm

The real reason is that this is stuff I have laying around already.

It is a good question and I will look into physical integration. I have a lot of arduinos laying around too but I’ve never tried to integrate with my iPad (besides WiFi). I am not a member of the Apple mFi program so physical integration with an iPad I think is not that simple. But maybe things have changed since I last looked at it.

Yes BT integration can be tricky with iPad and ESP32. I wrote a little iPad app to make this demo work so the iPad would accept it as an HID. My iPad is old so I think if it works with this version then future versions of iPad should be easier/better.

I have 5 buttons working just fine so far but am planning to test out a 5 way analog rocker tonight to see how that goes. My plan is to make it drive the little orientation cube. (Which is what my 5 button config currently does). It is pretty limited what actions can happen with Shapr3D right now but I expect they will add more key binding as time goes on.

The prototype is currently running off a rechargeable battery and/or can be plugged directly into USB to charge while using it. I for sure lean towards wireless for something like this but I have nothing concrete in mind yet. On my laptop my mouse is BT so seems pretty reasonable to use BT. But honestly, I am currently just using the path of least resistance to see how usable something like this can be with a primary focus on the form factor of the human-to-machine interface. Processors, code, etc I think is a distant second to that at the moment for me.

One thing I’ve not come to terms with is to keep this as a DIY level or go for super-efficient custom hardware. I like the idea of the DIY level.

If you get the ATiny thing working I would love to see what you do there.

Yepher · February 24, 2021, 4:36pm

I had a look through your code in that gist. Looks to me like you are missing several important elements for supporting BLE keyboard with iOS devices. Am I missing something?

Nitro · February 24, 2021, 6:29pm

In all honesty, I have to say that the iPad itself should be used more ergonomically, but by no means tinkering with additional buttons.
I myself would never want to use such buttons, that contradicts the simplicity.
The touchscreen itself should be used intelligently:
An example:
If one finger is motionless on the orientation cube, a list should scroll down where you can select 6 commands (simply swipe your finger down to the command you want) that you have devised yourself once in the settings.
I think that would be a very elegant and simple way of executing your own commands.

Nitro · February 24, 2021, 6:09pm

Addendum:
You could also add a little finger button under the orientation cube, with which your own command list scrolls down without delay!
The whole thing is so very elegant and defies simplicity!

milouchien · February 24, 2021, 6:51pm

A second is a long time for an impatient artist!