Hi there 👋 I’m Marco, VP of Product at Switchboard. AMA 👇

@chrismessina Hi Chris! When it comes to validation, I think we can break this down into a few parts... when I first joined Switchboard, the company had built a product that was focused on making it easier to get high quality guitar lessons online (for real!) and was starting to think about how this could actually be a much bigger opportunity in the context of work. The stage had been set with an investment in the core cloud-browser technology, but a different focus was on the horizon. (1) The initial "validation" we did shortly after I joined was to interview a *lot* of folks working remotely during the pandemic to understand how it was going. What was good? What was missing? How were they getting things done (or not) as a team in the new world we were all suddenly living in. Those discussions help form the initial set of hypotheses that pushed the product vision forward and and set the initial roadmap in motion. (2) Once we had something (barely) working, we immediately gave it to a number of teams to try in a small private beta. We spoke to these teams regularly to understand their experience with the product. This helped us prove (or disprove) some of our initial hypotheses and allowed to us further prioritize what we needed to work on before adding more customers. (3) That leads us to today! A month after our PH launch (thanks again for hunting us). We've seen an incredible response from the market in terms of people signing up for the product and using it with their teams. The validation we are seeking at this point is making sure we get to product-market fit – to make sure we build a product that people love and want more of. We're approaching this through the well-know framework outlined in Rahul Vohra's piece: https://review.firstround.com/ho... . Even after only a month, we've issued our first PMF survey and the results are helping us focus on the things that will be most important to our customers. In terms of optimizing for everyone's productivity, living on Switchboard as a company has really helped make that possible. We have rooms set up for all of our recurring meetings, projects, and external collaborations and each of the rooms keeps all of the materials we need organized and ready for the next discussion. We've fallen into a great weekly cadence as a team and that repeatability helps anchor us as we move forward together.

@aaronoleary Thanks for the question. At the beginning, there was a good sense of the initial set of use cases we wanted to support, but the most important aspect was ensuring that speech recognition, natural language understanding, and text-to-speech were at a level that made the envisioned experiences possible. Back then, the core technologies that powered those "bread and butter" systems were all in a very different place than today. That combined with the scarcity of usage data to grade and use for training purposes made the cold start problem challenging, and required a company even as large as Apple to get scrappy when it came to assembling the training and test data required to launch the product. It sure was fun :) We definitely imagined the voice assistant going much further than where many of the leading products are even at today. It's still early days (even though its been 10 years!).

@yuryfication Thank you for these thoughts. Traditional videoconferencing products made it possible for all of us to keep moving forward in the midst of the pandemic, but those tools are primarily focused on communication vs. collaboration. Sure, you can screen share, but as you pointed out, it's mostly about people "witnessing" vs. doing. In the "before times," if we met to get something done together, we wouldn't stare at each other's faces -- we would have a computer, a whiteboard, or a few different sources of shared context. With Switchboard, we're trying to provide that sense of context and collaboration by making it possible to work seamlessly and side-by-side in the same way we used to. This makes time together feel less "performative" and more about getting things done.

@dudejonathan041 Thanks so much, Jonathan – I'm really glad you like the product. There were definitely a number of large architectural, technical, and UX challenges that we had to overcome.. and sometimes in the moment you might stop and say "wow - are we going to be able to solve this one?" That happened a few times along the way, but we were able to make it to the other side. One thing we struggled with was how far to push before we opened up the product for the public beta. We ended up deciding to spend a bit more time on some very technical projects that improved stability of our cloud browsers, but it was absolutely the right call.

@jaythesong Hi Jay! Good to see you here! We use a lot of great tools (aside from Switchboard, of course 😉). Here's a rundown of some of them: 1. Figma for product design 2. Linear.app for roadmap planning, bug tracking, sprint planning, etc 3. Loom for asynchronous demos and design reviews 4. Gong to record and analyze customer interviews (hopefully it will integrate with Switchboard soon) 5. Segment + Amplitude + Snowflake + Metabase for our core analytics stack 6. Notion + Google Suite for all of the docs 7. Canva for quick and dirty marketing design needs 8. Wynter for testing messaging and positioning 9. Descript for editing audio/video content 10. We also have a Salesforce instance in place and we're just getting it up and running Also, this isn't a tool per se, but @chrismessina recently taught me how to make animated gifs with Keynote and I'm completely obsessed.

@manjot_pal Hi Alex. Definitely more challenging these days, but there are a few things that help us. Also curious what you find useful! 1. Gong - this product makes it easy to record customer interviews and analyze the transcripts. You can add tags and it's really handy. I'm a big believer in talking to customers as much as possible and tools like Gong make it very easy to take action on those discussions. 2. Typeform - we're using this for all of our PMF and other customer survey needs. 3. Wynter - this has been a really valuable tool for testing messaging and positioning. you can run tests very easily and the turn around is fast. 4. LaunchDarkly - we use this to manage which features are live in production. It allows us fine grain control over who can see the feature, so we have the ability to turn it on for specific customers if we want to run a test or do some research about a specific feature or interaction. 5. For data/analytics we're mostly using Amplitude. When I hit a wall I end up writing SQL in Metabase. (I secretly yearn for these moments 😂) 6. We're contemplating tools like usertesting.com as we grow as a way to get quick feedback on experiences we are designing. There's also a lot to be said about the power of an interactive Figma prototype. It's so easy to create high fidelity mockups that communicate an intended UX.

@ireteh Hello! I love working with freelancers for specific projects or in some cases a staff augmentation capacity (especially while searching for the right full time hire). One thing I probably wouldn't personally rely on a freelancer or outsource is Product Management. It's just too core to the success of the product and company. There might be very targeted projects where it could work, but I wouldn't outsource your product vision or roadmap -- you want your product managers to be deeply integrated into your teams and have skin in the game.

@iacono yeah, that was my thinking as well. I'm really enjoying currently outsourcing functions like Marketing, SSM, PR, UX but my first hire will probably be a PdM that will join for the long run. Thanks for your quick response!

@seansong great question, Sean! (1) There are a lot of metrics you can define and measure up and down the user experience of a voice assistant (e.g. word error rate for ASR, confusion matrix for machine learned NLU, etc), but to me, the most important is "Task Completion Rate." That is, if the user asks the assistant a question, did they get what they were looking for. This requires grading the responses and associated UX, but almost always uncovers issues in each of the different systems that work together to create the core experience. (2) Could you provide an example of something that might require this kind of boundary/tuning?

@iacono Appreciate your reply. For (1), got it, it is the "end to end" measure of a VA system, Crystal clear. (2) At early stage of VA dev, objective test guarantees its basic performance (usable and reliable), then subjective test improves its performance. With increasing knowledge territories (from news to music, from music to restaurant, then to maps, etc.) we cannot always rely on increasing human resources for product verification, so we need to do more objective test.

@ksdme You're welcome – this has been a lot of fun. I think of feedback as a gift, and if I keep getting the same gift over and over.. that tells me something :) It's very important to validate that your idea has both a market and that it serves the needs of that market. As a product manager or founder, you need objectively balance when it's time to keep pushing on a particular piece of that vision vs. when you have sufficient evidence to the contrary or evidence that your prioritization should be adjusted. Talking directly to customers, issuing surveys, and finding ways to get in-product feedback are all valid, but oftentimes it's much easier (and faster) to act on this data at a small company.

@khalidm3 Hey, thanks so much! I appreciate it. What's really interesting is that the same set of use cases rise to the top of usage on mobile devices, no matter which voice assistant you are looking at. I found that for tasks that took multiple taps/interactions and were pretty repetitive, people prefer voice, while tasks that are accessed very quickly are generally better candidates for touch input. Another way to think about it is: if it's already on screen, touch is easier -- if it isn't on screen, voice is easier. (this might seem obvious, but you'd be surprised). I'll check out your app!

@ryangilbert Great question. I actually transitioned from Apple to another small startup that was acquired by Samsung. That transition was definitely a big shock. I went from having a very specific focus at Apple to a situation where I pitched in on many different things across the company. For example, in my second week at Viv Labs I started a "new employee manual" because it didn't exist yet! I was also able to help on business development, get much deeper into design, and eventually ended up running much of the company. I liken my time at Viv to getting an MBA... you just get so much more exposure to all aspects of a company when operating at a smaller scale. Going from a post-acquisition company (working very closely with Samsung) to Switchboard was less of a shock. Going in, I had some intuition about what to expect, what kind of processes would or wouldn't be in place, and I had a strong sense of excitement to jump and and move quickly again. The biggest challenge was starting a new job without being able to meet my colleagues in person and figuring out how to build relationships, trust, and culture in a fully remote company. Switchboard (the app) has made these transitions even easier for people who have joined the company recently, because it makes spontaneous chats so easy.

@shovan Hi Shovan 👋🏼 While I think a lot of this depends on the stage, scale, and nature of the product, I've found through my experiences that when your QA resources are very limited, living on the product as much as possible makes a big difference. There's also a lot that can be done on the engineering side to put automated testing and CI/CD integration in place while ensuring that quality and testing is part of the core engineering philosophy itself. I've also been known to host a bug bash or two :)

@jaydev13 👋🏼 It's sometimes very common for companies to think that simply adding more features will make customers happy, but I've found that in almost every case that isn't true and in talking to your users, you actually discover the most important things to focus on. In the early days, I would make sure to build a great user experience centered around the core value of your product and start talking to your customers/target personas right away. If you mean focusing on adding new features vs. pure user growth, I think it's always important to get enough users on the product that you can validate your assumptions and get a sense of PMF before really focusing on user growth as *the* primary goal.

@iacono This is so perfect. Exactly what I believe and am implementing in my startup. It's all about continuous evaluation by features and improvement. Thanks for the words :)

@ryangilbert Ooh good one. Everyday I look forward to seeing our vision come alive in the asynchronous design presentations that are ready in my inbox when I wake up (my designers are in Europe and South Africa at the moment) and in the updates that our engineers add to our internal Switchboard environment everyday. Just last week I logged in one morning and I could suddenly change the color of sticky notes (which wasn't possible the previous day) -- it was an instant smile at 8am! What I least look forward to is jumping on traditional videoconference calls and accidentally trying to scroll on a screen-share only to realize I can't because it isn't Switchboard (seriously, it bums me out!)

@sobia_sheikh Probably a lot of great ways to answer this question, but one simple way that I love is to look at how many duplicate/similar issues have been filed as requests from customers. We used to do this at Apple as well.

@iacono Thanks. Talking from a startup product perspective when there are not customer and we are still in discovering phase .. so among features we have to see what should be considered ... Although some factors like market/competitive analysis goes but if it is new concept.. is hunch cool to go or we should add numbers based on vote where most believe trend would go.. One way is to just validate each concept with the trial persona in hand and build concept over there... But even for a feature.. some decision to be made based on the scope or depth of that features... what should be tested first.. what to put down.. so asking any framework or something companies like apple using to measure impact in discovery phase

@saniiliya Hi Sani! I've found that people really love a few things about Switchboard: (1) The permanence or "meeting memory" of each room. people can create dedicated rooms for recurring meetings or projects and keep all of the relevant files and docs open in the room for the next meeting or even in between meetings (2) The ability for each participant to interact with the cloud browsers and other content in the room. I think a lot of the frustration people experience with screen-sharing comes from their inability to participate. (3) The focus on content instead of faces. If we were working together in person, we wouldn't be staring at each other's faces. We would be using a laptop, working on a whiteboard, or even marking up some paper. Switchboard gives people that same feeling where the important context is the shared content, not faces. (4) Not having to worry about screen-sharing. When you can open all of the web-apps you need right inside of the room, you don't need to worry about oversharing your desktop, incoming notifications, etc.