Handycap Computer Usage
This is a program I wrote back in high school. The idea came from Johnny Lee and what he did with the WII mote (http://www.youtube.com/watch?v=QgKCrGvShZs). I found more resources for filtering infrared light so I could use my webcam instead of a WII mote. Turns out that blank film negatives filter infrared light, so I rigged a couple of layers to fit over my laptops webcam. About a week or so after writing the program I was selected to do an in person survey with the super attendant of the schools along with a few other students. One of the students was a quadriplegic. The schools in my area are trying to implement technology in their curriculum and the handicapped student complained that the schools are trying to encourage the use of technology but he wasn’t able to use it. After hearing him talked about not being able to use a computer since his accident I decided to show him my program and offered to make a set-up at his house so he could use a computer. I took an old hat and rigged an infrared led to it along with a wireless mouse; I extended the click button down to his mouth. He has been using this program for about a year now and is surfing the web, sending emails, and watching porn like the rest of the world.
Requirements: =-=-=python (python.org) =-=-=PIL (http://www.pythonware.com/products/pil/) =-=-=VideoCapture (http://videocapture.sourceforge.net/) =-=-=pygame (http://www.pygame.org/news.html)
#Webcam IR mouse control 1.6
#Tech B Sept. 9, 2009
#Works only with an IR filter on the webcam.
#To do this simply cut out the black part of a film
#negitave, and place two or three layers over the lens
#of the webcam.
######################################################
#----------imports and global variabl assignment------
######################################################
from VideoCapture import Device
import Image, sys, pygame
from ctypes import *
import ImageOps
from PIL import ImageEnhance
from pygame.locals import *
cam = Device()
user = windll.user32
res = (1440,900) #set to the resolution of the screen
pygame.init()
screen = pygame.display.set_mode((640,480))
pygame.display.set_caption('IR Mouse Control')
font = pygame.font.SysFont("Curier",26)
#---------------functions-----------------------
################################################
#Returns the xy cordinets of the ir dot.
#Doesn't return exact dot, only the first TRUE pixel value
def xy(im):
imxy = im.getprojection()
imx = imxy[0]
imy = imxy[1]
x = imx.index(1)
y = imy.index(1)
return (x,y)
#Returns size in pixels of the ir dot
#Was going to be used for click event
#No longer needed, but I think its a cool feature =]
def irint(im):
xyi = 0
irxy = im.getprojection()
irx = irxy[0]
iry = irxy[1]
for i in irx:
if i == 1:
xyi +=1
for i in iry:
if i == 1:
xyi += 1
return xyi
#-----------Main loop--------------------
#########################################
while 1:
for event in pygame.event.get():
if event.type == pygame.QUIT: sys.exit()
try:
imt = cam.getImage()
im = imt.resize(res)
im = ImageOps.mirror(im)
im1 = ImageEnhance.Contrast(imt).enhance(1.0)
im1 = ImageEnhance.Brightness(imt).enhance(1.5)
x,y = xy(im)
xyi = irint(im1)
user.SetCursorPos(x,y)
irpix = font.render("IR intensity in pixels:" + str(xyi), True, (250,250,250))
name = font.render('By Tech B', True, (250,250,250))
web = font.render('Webcam IR mouse control : 1.0.1', True, (250,250,250))
fil = font.render('Works only with an IR filter on the webcam lens', True, (250,250,250))
im1 = pygame.image.frombuffer(im1.tostring(), (640,480), "RGB")
screen.blit(im1, (0,0))
screen.blit(name,(0,26))
screen.blit(web,(0,0))
screen.blit(fil,(0,52))
screen.blit(irpix,(0,78))
pygame.display.flip()
except ValueError:
pass
Like I stated before the clicks are with a hacked mouse. I tried doing the click with code, but couldn't find a reliable solution. Some draw backs include CPU usage is high and other infrared light can effect the mouse such as a cigarette or open flames.
The next step is to lose the IR filter and base it on facial recognition. I've seen it done, but don't quite know where to start. Any suggestions or criticism is welcome, and feel free to tweak and distribute the code.
wolfmankurd wrote: If I understand this correctly and he is indeed using your computer/hardware to do this shizzle. Then seriously top notch stuff. I agree. The experiment on its own is of interest. The fact that it also can be put to good use is a plus.
I hopped on ye olde search engine and found a number of results but, since I am not more than a (barely) Python dabbler, I can't judge the usefulness of the resources. However, it looks like there are a couple of Python libraries for face recognition, and there's also this blog post that is, if not what you're looking for, full of links that give you more places to look.
I'd love to hear how further experimentation goes, too. :D
I don't have any pictures of it. But I can describe it. Its a black hat that has spots of white paint on the side. The IR led is in the center of the bill on the very edge. there is a switch from the battery back on the right side of the hat for the led. The mouse is velcroed on the other side with the face off of it. And a big gauge wire leading from the left mouse button down to the mouth. I used a push button switch for his button, its wrapped in shrink wrap to keep the saliva off of it.
total cost with out the mouse, $15. with the mouse $45.
I had most of the items already though. I used to mess around with DC circuits and microcontrollers. BasicStamp2.
And thanks for the links.
UPDATE:
I can now read two IR dots. I'm thinking about basing the mouse movements on the slope of the two dots.
#hopeit with two IR points
#mouse control proto
#Tech B.
#d.bmp and d2.bmp are two 7X5 images to show the two
# seprate points.
import VideoCapture
from VideoCapture import Device
import Image, sys, pygame, time
from pygame.locals import *
from psyco import full
from ctypes import *
user = windll.user32
full()
cam = Device()
pygame.init()
screen = pygame.display.set_mode((640,480))
font = pygame.font.SysFont("Curier",26)
d = pygame.image.load('d.bmp')
dT = pygame.image.load('d2.bmp')
def xy(im):
imxy = []
x = 0
y = 0
while x*y <= 307200: #Resolution of the cam. 640*480
if im.getpixel((x,y)) >= (250,250,250):
imxy.append((x,y))
x += 1
if x == 640:
x = 0
y += 1
if y == 480:
return imxy
x = 0
y = 0
while 1:
try:
for event in pygame.event.get():
if event.type == pygame.QUIT: sys.exit()
imc = cam.getImage()
bg = pygame.Surface(screen.get_size())
bg = bg.convert()
bg.fill((0,0,0))
dots = xy(imc)
if dots:
imc = pygame.image.frombuffer(imc.tostring(), (640,480), "RGB")
bg.blit(imc, (0,0))
bg.blit(d, dots[0])
bg.blit(d, dots[-1])
screen.blit(bg, (0,0))
pygame.draw.line(screen,(255,255,255), dots[0], dots[-1])
pygame.display.flip()
else:
pass
except IndexError:
pass
Featured on HAD they never actually use stuff I submit :)