Want to identify plants with your smartphone?

posted in: Science | 45

Editor’s Note: We have been receiving a large number of requests to identify plants for people. We don’t have the resources or expertise to be able to do this. Please contact a local university extension expert if you are unsure about a plant that you are looking at. You can check up with iBoPlanet™ to see if they have completed their app, but please do not ask us to ID your plants! Please enjoy the interview below. Thanks!

Recently, we received a message from an enterprising Floridian who wants to help people identify every plant they come across – with nothing more than a smartphone and a camera – and an application that he is building. His name is Steve Bowen, and he is trying to raise money for his smartphone app, called iBoPlanet™. He explains it in detail with a video on his Kickstarter fundraising page. The basic idea is that if you had a plant you wanted to identify and learn more about – all you would need to do is take a photograph of the leaves and iBoPlanet will match it to its database of plants and tell you the name, species, and some nutritional or environmental details about it. Add in the ability to tag your plant find with your location via GPS and see other people’s finds, and the plant-oriented folks out there might find this useful for research, ecology, and for educational purposes!

Being a plant-oriented person myself, I was intrigued, and so I asked Steve some questions about his project. Having taken a floristics course while an undergrad at UC Davis, I know how difficult it can be to key out and identify plant species, and that process often involves the use of flowers. We had a good conversation about it, which I shall present to everyone here to read and consider:

Me: In a nutshell, how does/will the app identify what species of plant someone is looking at?

Steve: From a high level, this will be determined through 1 of 2 ways:
1) the optical image technology will recognize exact dimensions, shapes, lobes, sinus, veins, petiols, midribs, margins, and leaf blades and cross reference that image with an open source database created (wikipedia-style) by users who have the app.
2) the optical image technology will recognize exact dimensions, shapes, lobes, sinus, veins, petiols, midribs, margins, and leaf blades and cross reference that image with databases of images of hundreds of websites through what is referred to as API (accessing existing databases) or RSS feeds to determine the species of plant or tree.

Me: Many species of plants look very similar on their leaves, and botanists will turn to identifying the species based on their flowers or other morphologies. Will your app be able to identify these plants all the way down to the species level, or what will its limitations be do you think?

Steve: At this point, we don’t see this as being too much of a problem but you raise a good point. However, the app SHOULD be able to determine all the way down to the species level and is the primary reason for the development of the app. We need to be able to not only tell not only that it’s a “Maple Tree” but whether its an Amur Maple or a Nikko Maple – for example and the optical image technology should be able to determine this. There MAY be a margin of error + or – 5% but we will be able to provide an open source gateway for that error to be corrected by Scientists correcting the issues (These “Scientists” will, of course, be vetted)

Me: If the program is confused about what species you are trying to identify, is there some way it will indicate that confusion, or will it decide on the closest match to species? For instance, if it can determine if the plant is a Maple, just not which species, will it just say that it is a maple or pick one of the species?

Steve: This is an excellent question. there WILL be algorithms in place (kind of like Google Goggles ™) that will be able to differentiate to within a degree of certainty. At this point of the design phase, we don’t know whether we will allow the algorithms to calculate “percentage of likelihood,” provide 2 or 3 results and then allow you, the user to pick the image that most closely resembles the image, or calculate probability and then conduct a 2nd cross-reference through the API/RSS feeds to drill down further to get closer to a 100% match. I am still working with a programmer who is highly regarded in technological mathematical algorithms to determine the best course of action. The project site at www.i-bo-planet.com will provide these types of updates as time goes on. Even after (if) the project is fully funded by its deadline. NOTE: If the project is NOT fully funded we receive $0 but, if we are successful, updates will continue throughout the software development life-cycle (SDLC) of the project until launch on the iTunes distribution platform.

Me: You mentioned that you would have the nutritional or medicinal value amongst the information presented – what will be your source of information for these characteristics? What if a mis-classification puts someone in harm’s way based on the information you present?

Steve: An even better question as “Safety” and “liability” will obviously play a huge part in this app’s conception. It is for this very reason that we are still determining whether or not the app should be “open source” (users upload their information after they are vetted to create our own database), access API/RSS, or both.

Me: Do you know at this point, or can you describe what the source of information would be for nutritional or medicinal value of the plants identified by the application?

Steve: The sources for this information are virtually limitless given the nature of how we will be able to access the informaiton through API and RSS Feeds. However, we will limit our sources of data aggregation to those of the highest repute and who are recognized by the botany/plant and tree genetics/horticultural communities. Perhaps you could make some recommendations? This is an organic process (no pun intended) and will evolve constantly to perfect after we launch. At this point, some sources we are considering are:

1. University Botanical Departments
2. Reputable blog sites
3. Botannical Gardens public domains
4. World Agroforestry Centre
5. KEW
7. Any other reputable public domain website
8. Open source options for professionals, such as yourself and you readership to play an active role in providing content. This will also apply to all other features of the app and the resultant website.

I appreciate your excellent questions. Moving forward I hope this is a lesson not in futility, but a learning platform by which we can utilize technology to further the stimulus of botanical/horticultural/eco education. Even if the app is 90% effective, through open source technology the 10% variance can be overcome and we can create a global community of interactive enthusiasts about the plant and tree life that make up our world.

So there you have it – a deeper look into this proposed project. Steve is looking for comments and suggestions for his project, and hopefully also some donors. There are about eleven days left in his Kickstarter campaign and a long way to go. Feel free to talk about it, ask questions here (and over there), and maybe someday there will be a way for every one of you to find out what that odd little plant growing in your back yard actually is!

Follow Karl Haro von Mogel:
Karl earned his Ph.D. in Plant Breeding and Plant Genetics at UW-Madison, with a minor in Life Science Communication. His dissertation was on both the genetics of sweet corn and plant genetics outreach. He recently moved back to his home state of California. His favorite produce might just be squash.