Embed Size (px)
Citation preview
ProgressReportQ3&Q4BrainControlforALSPatients(ATC20610)
AVoiceforPeoplewithALS
Prof.PeterDesain,PhDEstherSteenbeek-Planting,PhD
MarziehBorhanazad,MSc
September2018 The NoiseTag BCI consortium gratefully acknowledges the support for Assistive Technology of the ALS Association and the ALS Foundation Netherlands (18-SCH-391).
2
NoiseTag BCI consortium Radboud University, Faculty of Social Sciences, Donders Centre for Cognition P.O. Box 6751 6503 GG Nijmegen The Netherlands Phone: +31 6 51 888 767 (Prof. Peter Desain, PhD) E-mail: [email protected]; [email protected]; [email protected]
3
EXECUTIVESUMMARYPERSONNEL
Some movement of appointments between the Donders Institute and MindAffect took place, plus a few new hires (as of September 2018). A few appointments are in both organizations, supporting the process of knowledge transfer. Below, we provide an overview of everyone involved.
COLLABORATIONS
4
OVERALLPROGRESSOFTHEPROJECTWITHRESPECTTOITSCONTENT,SUMMARY In the agreement between the ALS Association, ALS Foundation Netherlands and the Radboud University, milestones and specific goals were formulated for each quarter. Overall, we are heading towards the main project goal, that is, of having a reliable BCI for communication available for ALS patients. However some internal project sequences were swopped as this worked out better: In a few goals there is a delay, and a few milestones planned for later quarters have been reached earlier. Below is the detailed report on this progress, in a rather technical style. The situation and scope has changed somewhat, like the decision to design and build our own headset, as no appropriate one was available. And moving towards a fully embedded system has become more ingrained in the core of the project. Furthermore the spin-off MindAffect makes it indeed possible to put more push on technology and plan for a full system with dedicated ultra-low-power chip. To reflect these changes and make use of opportunities, we propose to create a new milestone plan for Q5 - Q8 and propose it to the funders for approval. Also we would like to present our progress in some more detail and discuss it in person. The Glasgow meeting in December 2018 seems an ideal opportunity to meet.
FINANCIALACCOUNT
Financially, the project is on track. All personnel needed has been hired. Esther Steenbeek had to leave as of September 1st 2018, because of a busy other project, and she is replaced by Marzieh Borhanazad. For material costs, as we are running ahead of some of the future topics (see arguments above), and based on preparing software, hardware, and the zero-train function that is suitable for ALS patients, we decided to spend already some future costs for ‘Materials’. However, for the long run, we will stay within budget. One of the planned collaborators, Cognionics, could not deliver our required customizations of the EEG Amp at the low volume required for the first patient test at home, they agreed to withdraw. We will spend that part of the budget with another supplier (testing ANT amps and electrodes at the moment) and use some for extra (student assistant) work on evaluation. Later Cognionics may come back in for next versions in the scale-up phase.
5
REALIZATIONOFTHEQ3&Q4MILESTONES1. 1METHODINTEGRATION 2.
Detection hardware (Q3) The detection hardware comprised a Windows machine running Matlab, with a manual start-up sequence. This is the version that will be used in the hospital tests. The hardware has moved now further towards a turnkey system that can run on a mini computer, without screen or keyboard. This is the version to be used for the patient test at home. As next step the algorithms have been analyzed and subsequently translated to C and test-benched for an ARM M4 processor (technical report available). With a co-processor this can be chosen now for the next step in scale-up. The design for a single ultra-low-power chip of this setup has been created, according to our specs. This is ready to be send to the chip-manufacturer this fall. Dynamic Codes (Q4) Codebook transfer between detector and presentation machine in two directions has been implemented (over Bluetooth). Experiments with different codes have now becomes very easy. Truncated codes (consisting of only one kind of event), long codes (to improve generalization behavior for 1-class training), Gold codes (vs random), and simulation of different frame-rates have all been tested and informed our decisions on what to use best in the BCI. Machine Learning (was planned for Q5) For the machine learning, the early stopping rule (that was designed and been functioning in a pragmatic manner) now received an analysis and formalization based on first principles. Not only stopping a trial and emitting a symbol is modelled but also stopping the train session when enough data is available, or stopping the unsupervised (zero)-training. This new approach has become possible by using an in-product similarity measure, instead of a correlation - easing the description of its behavior over longer and longer timespans.
3. BUSINESSDEVELOPMENT Reimbursement (Q3) The inventory and negotiations with health insurance companies has been put on hold. We are designing and proofing a scheme in which a small subscription fee would be enough to obtain the equipment for those who need it. In this disruptive scheme, buying and delay caused by the approval process no longer occurs. We would like to investigate if the ALS associations can play a role in organizing this. If we fail to make this happen for some reason, we will go back to the traditional reimbursement route. Countries (Q4)We have built the strongest contacts with Belgium. But also Japan, France and the UK are connected. For these we will need (adapted) CE certification. After the patient tests will have proven the quality of the communication tool, we will aim for USA and need FDA approval. We are considering if it would be feasible to include US in our pilot tests with patients at home planned for Belgium and the Netherlands.
4. PATIENTTESTATHOME Headset test (Q3) We had to create our own new headset design because not what we require is not available. NPK took care of design and prototyping. We tested the possible electrode locations extensively in in simulation based on collected very high density data. Analysis Report (Q4, delayed)The test in the hospital will start in September, causing the analysis report to be be ready before the end of the year. The tests at home are scheduled for start 2019. Analysis of this study can only be expected in Q6.
6
5. EEGHARDWARE Headset production (Q4) Production of the whole headset is too early, but preparing for production of parts not
- Montage, with NPK - Electrodes according to out center-surround design, with
Datwyler and ANT - Amplifier, ANT is replacing Cognionics as they cannot deliver
according to the budget at the desired low volume. ANT is willing to invest in the joint development of Amplifier and Electrodes.
Please note: information in this figure is confidential!
6. PATIENTTESTINCLINIC Prepare Test (delayed Q1) Preparation is done. The tests were extended with a simple Yes/No condition for patients that achieve low performance on the speller. This task is easier and expected to function well in later stages. Forms for subjective rating (of e.g. fatigue) have be made better. Analysis report (delayed Q2) To be expected before end of 2018. Prepare Test Full (Q4) Because of a delay in the preparation, and because we judged that experimenting at home would yield much more insight for evaluating actual use, this task is moved to the patient tests at home. This experiment will measure, next to system performance and statistics of use, the headset comfort during long use, usability, effectuated change in communication patterns etc.
7. COMMUNICATIONPATIENTS
Interviews (Q3) Several interviews have been conducted, visiting patients or meeting them at events of the ALS community. Outcome is, among others, the felt need for access to social media. We are considering this extra feature. Another outcome is the desire for the least obtrusive flicker patterns, we are addressing this in the following way. The rate of stimulation may be doubled with the new model iPads. This improves comfort and retains system performance. The stimulation may be given by color-contrast. Experiments have been done, but were inconclusive, we are looking into this further. Further, the contrast may be under the patients control, trading increased comfort for reduced speed of detection. Feedback Headset Tests (Q4) As no proper existing headset with dry electrodes was found, MindAffect ordered a design agency to design and build one according to our specs. This is ready for testing as of mid September. First tests will be on comfort and of course on function. Demo Events (were planned for Q5, Q8) There have already been several events at which the system was demonstrated, like the ALS patient day. There is an active involvement of the MindAffect company and the Donders team with the ALS city swim event, at which there will also be a demo and info stand.
7
8. INSTRUMENTATION Stimulus Kit Design (Q3) Design and implementation of the stimulus LED control box has been realized and a protocol of communication via USB has been defined and implemented. Timing accuracy has been verified and is thoroughly tested. Stimulus Kit Release (Q4) The release of the stim kit, now that we create our own headset, makes not much sense without means to detect (needing the headset). This is postponed. The extra time was used to make the LED buttons wireless. LED Buttons (were planned for Q5) The prototype set of 8 LEDS is ready and can be used. The response turns out to be similar to stimulation by tablet (iPad).
9. EXPERIMENTS IP Paper (Q3) The Zero train paper has been prepared, comparing different ways of calibration/training. Experimental instrumentation has been made. A student assistant is hired to run the experiment with 20 subjects.. Adaptive Impedance (Q4) Because of our new headset design with low channel count, this has become less important. The feature will be subsumed in the fully adaptive version that can also deal with headset displacement, fatigue and other non-stationarities. An extensive planning of all experiments need has been made.
10. EMBEDDEDSYSTEM Detection box (Q3) This is the version that will run in the patient tests at home. Since our new octave and C implementation the detection box needs no longer to run Matlab. This frees the choice. Simulations have been made to check performance of ARM M4 processor. We are checking the feasibility of Raspberry Pie, to further reduce the cost. Design (was planned for Q5) Overall new architecture of the software has been designed and implemented, with the code factoring based on streaming (data + events). It is based on thorough mathematical formalization. Migrate Software (was planned for Q6) A C version of the new detector code has been made. It performs conform the Matlab prototype and is ready to be embedded in the processor. Initiate Chip Design (was planned for Q8) With the Technical University of Eindhoven a chip-design has been created. The tape-out date is planned for the end of 2019. This unforeseen opportunity opens a whole perspective of scaling-up and low-cost solutions.
