How we deployed Copilot for 365 [Long read]

If you are a tech enthusiast, you have probably heard of Copilot, the AI-powered assistant that helps you write code, documents, emails, and more. Copilot is a product from Microsoft and OpenAI, and it uses a large language model (LLM) to generate text and images based on your prompts, context, and data sources. Copilot is available as a subscription service for Microsoft 365 users, and it integrates with various apps such as Word, PowerPoint, Excel, Outlook, Teams, and more.

When the LLM (large language model) crazy started to break, we were super early adopters, with ChatGPT subs as soon as they came available, and Copilot subscriptions activated on about the same timeframe.

Copilot has been immensely helpful, and I’m often getting asked “how did you manage to get it set up so quickly”, and “how did you teach your users how to use it”, and “what benefit did it actually bring”?

In this post, I will share with you how we implemented Copilot at Random Creation House, a small company that I founded and run as a technical leader. I will cover some of the business case, how we did onboarding, and how we are using Copilot now. I will also share some of the challenges, tips, and best practices that we learned along the way. I hope this will be useful for anyone who is considering or already using Copilot in their organisation.

There’s something important to note here: I’m aware that you might be implementing Copilot into companies, enterprises, corporates, which may have a slightly different risk tolerance to us. Whilst it goes without saying, you really shouldn’t make this post your only stop… speak to you Microsoft technical account manager too 😊

Ok so to the good stuff.

As regular readers will know, I’m the Founder at Random Creation House, and I’m definitely in the “Technical founder” bucket.  I have ADHD, so I’m always looking for tools and systems that can help me through my day.  I came up through technology as a grassroots technologist: I spent my teenage years “educating myself on how one might break into IT systems” (there are no lines to read between on that comment…), I learned a bunch of programme languages as a kid, and over time I’ve taken all my technical curiosity, bundled that up with a bunch of programme management and project management skills, and have spent about a decade running really big technical programmes for corporate enterprise, as well as running my own small company.  As a result, my approach is often a weird mix of tech enthusiast, user-needs-focussed product manager-ish fact finding, and cold hard business case delivery… lets start there... and buckle up, it's a long read!

Business case

Let’s get it out there, Copilot isn’t cheap.  The list price on our Business Premium licenses are something like £18 per month, and the list price on Copilot is about £25 per month.  If you took those "per user per month" numbers out of context and asked my accountant’s advice, his words would be…. direct!

I wouldn’t ask that question, but I see conversations every day online about “how can it double our licensing cost?!”; the reality is, it’s such a new product, I don’t think anyone really knows what AI helpers should cost, and it looks like Microsoft, OpenAI, Google etc have all just settled on “the cost of a couple of pints in a London boozer”… we could argue cost, value, would the market be twice as big if the product was half the cost, etc, but in the end, the price is £25.

Now we’re a small outfit.  When I looked around, I almost lumped for “lets just get one for everyone” – the thought process being, the appx cost of one of our people is £30 per hour, just over £5k per month, in wages.  That’s before the cost of IT, licensing, space to work. If you look at the business case “this is £30 for a £5k per month asset (person) to be more efficient”, then the maths start to stack up better… the £30 Copilot license alongside a £60/month Azure Virtual Desktop alongside the £20 O365 Business license - £110 per month per person against a £5k means their tech is basically 2.2% of their total cost… I could take off Copilot to make their IT costs 1.6% of their total cost, but why?  We’re not “more than doubling the license cost”, we’re supplying a tool, making tech 0.6% points more in an individuals overall "total cost".

My challenge to folks was “I’ll give you it, but you have to make sure you use it, it’s got to become worth than £30 per month, to me, and to you”.

I try to imagine what would’ve happened if Picasso’s manager had said “sorry Pablo, the numbers just don’t stack up – why do you even need two paint brushes?  Or “easel-ey does it… you’ve already used two canvases today”.   

So I made a deal with my team. I said, I will get you Copilot, but you have to make sure you use it. You have to make it worth more than £25 per month, to me, and to you. You have to learn how to use it, how to trust it, and how to improve it. You have to give me feedback, suggestions, and ideas on how to make Copilot better for us. You have to be open-minded, curious, and adventurous. You have to be ready to embrace a new way of working, with an AI partner by your side.  Use it or lose it.  Turns out, everyone uses it.  A lot.

That £25 fee has been earned back many times over.

Technical implementation

We have a pretty vanilla setup, so our technical config wasn’t so hard.  Even though it turned out that way, we still ran through the standard design questions – what data flows is Copilot using, do we need to implement any QoS, does it have an latency sensitivity, what ports/protocols/IPs/FQDNs, do we need to block personal copilot, how do we protect data assets, etc.

This is important to us, because Copilot is extremely powerful.  I can dump an entire business case into a document, and start querying it.  I needed to be sure that Copilot wasn’t sending my data to Bing, like “Find something to supplement this document [full doc attached]”, otherwise there’s a chance our IP could make it into the public domain, or that we could breach the privacy of our customers.

CDP (commercial data protection) comes as standard for our licenses.  It keeps our data contained within our domain, and doesn’t mix it up with public data, and it's the reason we have a "work" and "web" view in the Copilot web app.

We’re not using Purview yet, but its on the cards.  When we’re working with customer data, we’ve got an AWS (Sorry Microsoft ☹) service that we store in.  Realistically, Purview’s a bit more than we really need right now. We just need to be aware of what documents we're working with, and make sure we're properly marking INTERNAL data.

I took a look at the Entra-only copilot enforcement, like injecting x-ms-entraonly-copilot into outbound headers (using a proxy to decrypt/inject/re-encrypt), or setting a DNS entry for www.bing.com to be a cname for nochat.bing.com, and setting copilot.microsoft.com to be cdp.copilot.microsoft.com, but we just didn’t need to.  In reality, if the company was bigger, I’d implement these policies globally as a first step, so everyone could access Copilot, get prevented from signing into non-Entra copilot, and then license up the allowed-group only, but this became a complete non-issue for us.  Working on the principle of Enable -> Adopt and all that. But we don't have enough people for it to be a problem…

In the end, our enablement was near enough:

1. Open 365 Admin Centre - Home - Microsoft 365 admin center

2. Billing -> Purchase Services

3. Choose Microsoft Copilot for Microsoft 365, add license numbers, subs length

4. Pay – see licenses appear in “Your Products”

And then day to day:

1. Open Microsoft365Admin on iPhone

2. Assign License

3. Select Copilot

4. Choose the user, and save

(Enterprise Admins reading this - I can feel you wincing right now, in a larger organisation, and/or orgs who want to automate license allocation through Service Request catalogues, you can assign the Copilot licenses to users through an AD group).

Copilot springs into action pretty quickly, with copilot.microsoft.com firing up immediately.  Most folks I tell them just to chuck a quick reboot, and when they start back up, MS Apps have the Copilot features available.

It’s kinda annoying that the better Copilot experience is in New Outlook, not Outlook – I’ve used New Outlook a few times to get access to Draft with Copilot, but I’ve got an ongoing issue getting my iCloud calendars to sync to New Outlook (works fine with Other Outlook/ Outlook Previously Known at Outlook, Outlook Classic?... gah I wish we could have version numbers back), so I do spent a lot of time either being able to see my Microsoft Calendars with the cool Draft With Copilot feature, or I can see all of my calendars (including my iCloud ones) without Copilot… which is super frustrating, because it’s so very nearly perfect.

For those who need more granular controls, you can follow the same sorts of steps we did – when we were onboarding the documentation wasnt great, so trial and error: at the most basic, hit F12 and start a network capture whilst interacting with Copilot, and you’ll see that the traffic is most 443 HTTPS and Websockets towards copilot.microsoft.com, Sydney.bing.com (Sydney, I think, might be a MS Internal name for Copilot), and a few others.  By no means a particularly complicated set of web service interfaces.

There’s a load of good technical documentation now, I think this’ll be a common go-to for enterprise: Manage Copilot | Microsoft Learn. Or just ask Copilot the answer (*always* ask for sources!).

Adoption

People who I think just get Copilot: project managers, programmers.  That’s almost the full list of the types of people who I routinely work with.  Hurrah.

Now we were all pretty familiar with ChatGPT, and we’ve all tried Bard, so we weren’t at a standing start with most of the team. 

For those who couldn’t get what it was, I ran a few in-person demos – I asked folks to talk me through a problem they needed to solve.  As they spoke, I typed what they said.  My normal demo involves plain old copilot.microsoft.com (Work context), and as part of the demo, I ask:

“Think about what you’re aiming for, then ask me to do that job, describing as much, clearly, about the output as you can.  You’re giving me a recipe, and you don’t need to convince me. Prioritise clarity over courtesy.”

For project managers and programmers this is second nature – we’re all used to stepwise refinement, definition of outputs, and so on, because that’s how work packages / work breakdowns are built, and that’s how modules / subroutines / methods are built (I acknowledge the roar of protest coming from programmers right now because of this generalisation).

I’ve had some great examples.  Most people start with poems. 

Them: “Make me a poem about my holiday”. 

Sound the klaxon – that’s a fail.  I don’t know about your holiday.  I usually run the prompt anyway, and I usually add some mischievous content to the request to maximise the awfulness of the forced “hallucination”… and boom, there’s lesson 1 delivered.

The lesson is: the AI does have some good guardrails and will usually tell you if it’s making something up, but it doesn’t always tell you, and it doesnt know when it's expected to stick to facts vs be creative (the "Tone" does change the temperature / creativity boundaries, but that's a lesson for later). 

Hallucinations (where the AI over interprets something… if you didn’t describe the holiday, it has no context, so it’ll make something up so it can finish the task) can be extremely convincing, because on the face of it, it could be entirely plausible. 

We can ground the AI (grounding means to provide it with reference data) either in the "Work" tab of the Copilot web app, in which case the "grounding" can be provided by your corporate data. You can also ground the model by simply giving it free-written context. Running a well grounded model with a precise (rather than creative) temperature/tone is how to get fact. Tweaking the amounts of grounding, and temperature, generates ideas.

Me: “Try again.  Tell me about the holiday, properly.”

Them: “Ok, make me a poem about the holiday I took to Bognor Regis.  We drove our red fiat punto, never exceeding the speed limit, and stayed in a hotel on stilts.  It was called Hotel California, which made me laugh, because it was nothing like the Eagles song.  Every day we had toast and jam for breakfast, and the hotel’s kitchen was broken, so they arrange for Deliveroo orders to be brought to us, and we had a £50 spending limit each!  I bumped into my friend Carlos at the supermarket, and we’d both bought a pack of eggs and a newspaper – its surprising how similar we are at times.  The highlight of the trip was an evening where we went to see Les Miserables at the Bognor Grande Theatre, the show was excellent, with Dave Grohl playing Javert, and Michael Ball as Valjean.  At dinner I ate a bowl of cold beetroot soup, apparently inspired by Budapest.”

Aside from the whimsy in that particularly fictitious (if you hadn’t already guessed) holiday, I’m going to re-prompt Copilot. 

“Write a poem about the holiday described below, aim for 2-3 paragraphs…”

“In Bognor Regis, by the sea,
A red Fiat Punto carried me.
To Hotel California, on stilts it stood,
Laughing at the name, as anyone would.

Toast and jam to start the day,
Deliveroo's finest, come what may.
A £50 limit to dine like kings,
Such peculiar, delightful things.

At the market, a friend I see,
Carlos with eggs, just like me.
Our lives in sync, a mirrored view,
In Bognor Regis, under skies so blue.

Les Misérables, a night at the theatre,
Grohl and Ball, none could be greater.
Valjean and Javert, in a musical fight,
Bognor Grande's stage alight.

A bowl of beetroot soup, cold and sweet
A taste of Budapest, a culinary feat,
In Bognor Regis, where memories are kept,
Of the holiday where joy leapt.”

I see the light of creativity switch on. 

Them: “Oooooh that’s great”. 

Me: “Want to hear it as a song?”. 

Them: “OMG YES”.

Me, to Copilot: “Turn the poem into a song, maximum 2 mins long.  It should be a slow blues song, in the key of A minor.  Include chords.  There should be a spot for a guitar solo in the middle.  Adjust the wording if it helps with the flow of the song, and make sure there's a catchy chorus.”

[Verse 1]
Am                E
In Bognor Regis, by the sea,
F                 C       E
A red Fiat Punto carried me.
Am               E
To Hotel California, on stilts it stood,
F              C          E
Laughing at the name, as anyone would.
 
[Chorus]
Am               G
Bognor Regis, oh seaside delight,
F               C          E
With pebbled beaches, pure and bright.
Am               G
From Butlin's fun to Hotham Park,
F               C          E
A perfect holiday, leaves its mark.
 
[Verse 2]
Am               E
At the market, a friend I see,
F                 C       E
Carlos with eggs, just like me.
Am               E
Our lives in sync, a mirrored view,
F              C          E
In Bognor Regis, under skies so blue.
 
[Guitar Solo]
Am / E / F / C / E (x2)

[Verse 3]
Am               E
Les Misérables, a night at the theatre,
F                 C       E
Grohl and Ball, none could be greater.
Am               E
Valjean and Javert, in a musical fight,
F              C          E
Bognor Grande's stage alight.

[Chorus] (Repeat)
Am               G
Bognor Regis, oh seaside delight,
F               C          E
With pebbled beaches, pure and bright.
Am               G
From Butlin's fun to Hotham Park,
F               C          E
A perfect holiday, leaves its mark.

Amazing.  The take away here – Copilot can fill in the gaps (I didn’t tell it anything about Hotham park), and it can follow the instructions.  It mostly caught the story I fed in.  The lesson: we’ve created something new, from story data that we manually typed.  We took a couple of goes, one to write the poem and get the general feel, and another to convert it into a song.  I usually use some additional prompts to tidy up the content. 

Lesson: Copilot can take what you give it, build its own ideas (Which, are really just ideas that it’s seen or inferred from its training data), and present content back to you.   You can use single go to get what you need, or layered prompts to get and refine what you need. The "Notepad" section on the copilot web app is the place to play with big datasets.

This usually goes on for a little while.  I love to think of creative new ways to prompt it, make up new stories, because yes its a tool, but its also this amazing machine, and it's fun to play with.

There are scenarios where we need to format content in a particular way – our programmers really like to receive requirements / stories in a format that works with their tooling, so once the song’s done, I’ll demo something like:

“Rewrite my idea into the format described below. 

My idea is: I need the widget to be blue, not red.  When someone clicks it, it should calculate the mass of the universe, and display it on an LCD screen embedded in the hyperdrive transfunctioner.  When this happens, the user should be informed of the price, which is calculated using the existing model multiplied by the value of gravity.

The format is: Headline (less than 10 words), Description (less than 150 words), user experience (50-200 words), reach (some users, lots of users, all users), impact (on revenue, from significantly damaging to significantly beneficial), confidence (of our ability to deliver, low to high, in 5 steps), effort (to deliver, in approximate number of days).  If the data isn’t present in my idea, mark that as TBC on your output.”

And that'll create:

“Blue Widget with Universal Mass Calculation

Headline: Cosmic Blue Widget Launch

Description: Introducing a revolutionary blue widget that transcends the ordinary. Upon interaction, it computes the universe’s mass, showcasing the result on an integrated LCD within the hyperdrive transfunctioner. Simultaneously, it calculates and communicates the cost to the user, employing our established model and gravity’s constant.

User Experience: Users encounter a seamless interface where a simple click initiates a grand display of cosmic calculations. The blue widget not only provides a visual treat but also imparts a sense of awe as the mass of the universe unfurls on the screen. Post-calculation, users are promptly informed of the price, ensuring transparency and enhancing satisfaction.

Reach: TBC

Impact: TBC

Confidence: TBC

Effort: TBC”

It all starts to fall into place from this point.  Do a few words saying what you want to see at the output, a few words around what you’ll use it for to give context, and any style guides – brief, concise, professional, casual, etc. Maybe this example doesn't resonate... I'll properly demo this soon. You could argue that actually the additional words detract from the point... and you'd be right.

Other examples

There’s only really one thing left – references (Aka augmented intelligence, or grounding).  In Copilot, you can drag images into the main interface, you can reference documents from Onedrive, or you can ask it to look at emails, teams messages.  Prompt something like “I've been away from work for a week. Review all of my emails, chats, teams messages from that period and provide a list of tasks assigned to me, in the form of a table, including the source, source document title or name, action, assigner, due date (if known), priority (if known)” and suddenly the response feels unnervingly helpful.

When I’m drafting an email, I’ll often use Voice Dictate to say everything that’s on my mind (into OneNote), and if I have a screenshot or photo with text in, I’ll paste that into Onenote too (and use the right click->extract text from this image feature).  I’ll either use the New Outlook “Draft with Copilot” feature to write the email, or I’ll use Copilot Notepad or Copilot Work, on a prompt like “The text below contains unstructured thoughts that I want to turn into an email, which shouldn’t be overly long, but needs to be comprehensive.  Organise the thoughts around themes, and propose a wording for the email, which I can send to Dave Daveson on the subject of ‘Thoughts since our last meeting’.  Review my recent Teams messages and emails to see if there’s anything that I’ve forgotten to mention”.  Bam, a reasonable “90% the way there” email usually pops out.

In Word, I’ll sometime use a prompt like “The reference document contains a set of data, and the template document contains a series of questions. Copy the template document here, and for each question, provide a response based on the reference document. Documents:

Template: /someTemplate.doc

Reference: /someReferenceDocument.doc”

The biggest help for me is that Copilot gets me near enough to the place I'd have got to anyway, and can take my non-linear thoughts and present those in a more logical way. It only "first draft", but a solid effort in most cases.

Whilst I could sit and write out a big long Excel formula from scratch, why not let Copilot do the first framework for me, give me ideas on how to tackle the data crunching?

Whilst I could refactor some linear powershell to use runspaces, why not let Copilot provide me with the framework, even have a first crack at switching the code around?

For inspiration, it's worth checking out the list of stuff that Copilot can do in 365 apps:

Microsoft 365 App	Feature	Description
Word	Draft with Copilot	Generate text with and without formatting in new or existing documents. Word files can also be used for grounding data
	Chat	Create content, summarize, ask questions about your document, and do light commanding via Chat.
PowerPoint	Draft with Copilot	Create a new presentation from a prompt or Word file, leveraging enterprise templates. PowerPoint files can also be used for grounding data
	Chat	Summary and Q&A
	Light commanding	Add slides, pictures, or make deck-wide formatting changes.
Excel	Draft with Copilot	Get suggestions for formulas, chart types, and insights about data in your spreadsheet.
Loop	Collaborative content creation	Create content that can be collaboratively improved through direct editing or refinement by Copilot.
Outlook	Coaching tips	Get coaching tips and suggestions on clarity, sentiment, and tone, along with an overall message assessment and suggestions for improvement.
	Summarize	Summarize an email thread to help the user quickly understand the discussion.
	Draft with Copilot	Pull from other emails or content across Microsoft 365 that the user already has access to.
Teams	Chat	Users can invoke Copilot in any chat. Copilot can summarize up to 30 days of the chat content prior to the last message in a given chat. Copilot uses only the single chat thread as source content for responses and can't reference other chats or data types (for example, meeting transcripts, emails, and files). Users can interact with Copilot by selecting pre-written prompts or writing their own questions. Responses include clickable citations that direct users to the relevant source content that was used. Conversations with Copilot take place in a side panel that allows users to copy and paste. Copilot conversations will disappear after the side panel is closed.
	Meetings	Users can invoke Copilot in meetings or calls within the same tenant. Copilot will use the transcript in real-time to answer questions from the user. It only uses the transcript and knows the name of the user typing the question. The user can type any question or use pre-determined prompts; however, Copilot will only answer questions related to the meeting conversation from the transcript. The user can copy/paste an answer and access Copilot after the meeting ends on the Recap page.
	Copilot	Allows users to access data across their Microsoft 365 Graph and leverage LLM functionality. Copilot can be accessed in Teams and when signed-in to Bing with an active directory account.
	Calls	Copilot in Teams Phone uses the power of AI to empower you to work more flexibly and intelligently, automating important administrative tasks of a call, such as capturing key points, task owners, and next steps, so you can stay focused on the discussion. Copilot in Teams Phone supports both voice over Internet Protocol (VoIP) and public switched telephone network (PSTN) calls.
	Whiteboard	Makes meetings and brainstorm sessions more creative and effective. Use natural language to ask Copilot to generate ideas, organize ideas into themes, create designs that bring ideas to life and summarize whiteboard content.
OneNote	Draft with Copilot	Use prompts to draft plans, generate ideas, create lists, and organize information to help you easily find what you need.
Forms	Draft with Copilot	Use prompts to draft questions and suggestions that help you create surveys, polls, and other forms with ease.

Conclusion

Copilot has been a game-changer for us at Random Creation House. It has helped us save time and effort, improve quality and consistency, generate ideas and insights, and have fun and satisfaction. It has also challenged us to change our mindset, our habits, and our expectations. It has also taught us to communicate, collaborate, and co-create with an AI partner. It has also enabled us to work more flexibly and intelligently, and to create more value and impact.  I’d almost go as far as to say, Copilot is my new search engine… it has pride of place permanently on my open-tab-list in Edge.

Copilot is not a magic bullet, or a silver bullet, or a bullet at all. It is a tool, a partner, and a culture. It is not a replacement, or a substitute, or a competitor. It is a complement, an enhancement, and a collaborator.

If you are considering or already using Copilot in your organization, I hope this post has been useful for you. I hope it has given you some insights and tips on how to implement, onboard, and use Copilot effectively. I hope it has also inspired you to explore and experiment with Copilot, and to discover and create new possibilities with Copilot. I hope it has also encouraged you to share your feedback and suggestions with Copilot, and to help Copilot learn and improve with you.

To find out more, do checkout the MS Learn pages, Microsoft's Copilot Adoption toolkit, and look out for the many great blogs and vlogs, or reach out to me over on Linkedin.

Thanks for reading, and happy Copiloting!