The improved lengthy context window may even pull data from a number of paperwork when responding to a single immediate. Within the aspect panel in Docs, I requested for assist writing a pattern letter to a possible job candidate — within the immediate I linked to the job description doc and the applicant’s PDF portfolio, each of which had been in my Drive — and immediately acquired a electronic mail draft, which factored in related particulars from each paperwork.
Gemini 1.5 Professional isn’t our solely shiny new mannequin, although: I additionally bought to attempt the freshly-announced Imagen 3, our highest-quality text-to-image mannequin but. One of many new skills I used to be enthusiastic about was its capability to generate ornamental textual content and letters, so I put it by its paces. I began by asking for a stylized alphabet — like letters spelled out in jam on toast, or with silver balloons floating within the sky. Imagen 3 generated a full alphabet of letters, which I might then use to sort out my very own (scrumptious) menus.
After my Imagen 3 interlude, I continued with extra Gemini demos. In considered one of them, I might pull up Gemini’s overlay on an Android telephone and ask questions on something on the display screen. This actually confirmed how we’re not solely increasing what you possibly can ask Gemini, however we’re additionally making Gemini context conscious, so it may anticipate your wants and supply useful solutions.
The use case right here was a prolonged oven handbook. Whether or not it is a demo or actual life, that is not one thing I would be enthusiastic about studying. As a substitute of skimming by the doc, I pulled up Gemini and instantly bought an “Ask this PDF” suggestion. I examined questions like “how do I replace the clock” and rapidly bought correct solutions. It labored simply as nicely with YouTube movies. As a substitute of watching a 20-minute exercise video, I requested a fast query about the best way to modify planks, bought a solution, and was on my method onto the subsequent demo, the place I examined a brand new dialog mode known as Gemini Live that allows you to discuss with Gemini within the app, no typing required.
Talking with Gemini was a unique expertise than the standard chatbot interface: Gemini’s solutions are much more conversational than the paragraphs of texts and bullet-pointed lists you may often get. In my demo, I realized you would even reduce off Gemini in the course of a solution. After asking for a listing of child’s actions for a summer season trip, I used to be in a position to interrupt a listing of solutions to dive in deeper on what supplies I’d want for tie-dying a shirt.
The Project Astra — or “superior seeing and speaking responsive agent” — demo took issues a step additional to indicate the reducing fringe of the place our conversational AI initiatives are heading.