This post is continuation on the series of posts dedicated to creating editor extensions in Unity.
You can find part one here.

Toolbars

In the previous part we mostly did preparation work, now it’s time to start drawing some stuff. But first, a few notes on GUI scripting.

GUI vs EditorGUI

In general using the EditorGUI/EditorGUILayout object in the editor windows seems to be better, as it usually draws controls that have a similar look and feel to the ones used in the editor. Also, EditorGUILayout.BeginHorizontal/Vertical returns the allocated rectangle during the Repaint event (more about the Repaint later), which will turn out to be quite useful. With that said, I haven’t found any reason not to mix between GUI, EditorGUI, GUILayout and EditorGUILayout

Drawing Toolbars

In order to draw a toolbar, all we need to do is draw a horizontal area using the toolbar style (either from EditorStyles or the editor skin). If you inspect the style in the editor you’ll find that it has a fixed height set at 18 pixels, which can be used as the height of your toolbar buttons. Keep in mind that you do need to set GUILayout.Height parameter if you’re going for the layout version, or there will be gap between your toolbar and the areas directly below or above it.

EditorGUILayout.BeginHorizontal (EditorStyles.toolbar, GUILayout.Height(EditorStyles.toolbar.fixedHeight), GUILayout.ExpandWidth(true));
EditorGUILayout.EndHorizontal ();

The lines above draw toolbar rectangle that stretches the whole width of the window.
The option GUILayout.ExpandWidth(true) is required to make the layout system stretch the toolbar rectangle, otherwise only space for the padding and border will be allocated.

Drawing the buttons is fairly straightforward as well – just draw a simple button using the toolbarButton style. A couple of things need to be mentioned – in the layout mode you don’t need to specify a height, since the button will take on the height of the enclosing area, but you need to specify width – for some reason buttons are expanding by default. Also, there doesn’t seem to be a toolbarToggle style, so drawing toggles (like “Collapse”, for instance) should be done using toolbarButton as well.

EditorGUILayout.BeginHorizontal (EditorStyles.toolbar, GUILayout.Height(EditorStyles.toolbar.fixedHeight), GUILayout.ExpandWidth(true));
{
    GUILayout.Button("Clear", EditorStyles.toolbarButton, GUILayout.Width(50));
    collapse = GUILayout.Toggle(collapse, "Collapse", EditorStyles.toolbarButton, GUILayout.Width(70));
    clearOnPlay = GUILayout.Toggle(clearOnPlay, "Clear on Play", EditorStyles.toolbarButton, GUILayout.Width(70));
    errorPause = GUILayout.Toggle(errorPause, "Error Pause", EditorStyles.toolbarButton, GUILayout.Width(70));
}
EditorGUILayout.EndHorizontal ();

You’ll notice one problem with the image below – it looks nothing like console toolbar, all the buttons are the wrong size. One solution would be to simply eyeball it or grab Photoshop and measure the exact size of the original buttons, but that’s not the proper way to do it.

Content Size

The best way would be to exactly measure the size of the content we put in the buttons using GUIStyle.CalcSize.

private bool collapse;
private bool clearOnPlay;
private bool errorPause;

private GUIContent clearContent = new GUIContent("Clear");
private GUIContent collapseContent = new GUIContent("Collapse");
private GUIContent clearOnPlayContent = new GUIContent("Clear on Play");
private GUIContent errorPauseContent = new GUIContent("ErrorPause");

private void OnGUI()
{
    var style = EditorStyles.toolbarButton;
    EditorGUILayout.BeginHorizontal (EditorStyles.toolbar, GUILayout.Height(EditorStyles.toolbar.fixedHeight), GUILayout.ExpandWidth(true));
    {
        GUILayout.Button(clearContent, style, GUILayout.Width(style.CalcSize(clearContent).x));
        GUILayout.Space(7);
        collapse = GUILayout.Toggle(collapse, collapseContent, style, GUILayout.Width(style.CalcSize(collapseContent).x));
        clearOnPlay = GUILayout.Toggle(clearOnPlay, clearOnPlayContent, style, GUILayout.Width(style.CalcSize(clearOnPlayContent).x));
        errorPause = GUILayout.Toggle(errorPause,errorPauseContent, style, GUILayout.Width(style.CalcSize(errorPauseContent).x));
    }
    EditorGUILayout.EndHorizontal ();
}

What we’ve done here is that we’ve created a GUIContent for each button, then we display that content in the button and also use it to calculate the required size of that button. Keep in mind that, as it says in the docs, CalcSize is not very good at determining the size of the content if it needs to be word wrapped.
Now, our buttons are the exact size they need to be, and coincidentally the exact same size as the original console (I admit to eyeballing space between the first two buttons).
There are, of course, much more optimized ways of using these functions, but since this is an example there is no point in making the code harder to understand. Also with this set up we can do something cool:

Custom Cursors and Resizing

As you’ve probably noticed, the original console is separate into two areas – let’s call the top one the message area and the bottom one – the details area.

Getting Area Size

To simulate that we’re going to draw to separate vertical areas, filled with to boxes to expand them.

GUIStyle box = GUI.skin.GetStyle("CN Box");
EditorGUILayout.BeginVertical();
GUILayout.Box("", box, GUILayout.ExpandHeight(true), GUILayout.ExpandWidth(true));
EditorGUILayout.EndVertical();
EditorGUILayout.BeginVertical();
GUILayout.Box("", box, GUILayout.ExpandHeight(true), GUILayout.ExpandWidth(true));
EditorGUILayout.EndVertical();

This looks very similar, but there is a problem – you can’t move the line in the middle, whereas in the original console the two areas can be resized freely. That creates a problem – we want to resize the stuff inside the main area, but we don’t know the size of the main area itself. We could probably setup some constants that ensure some fixed size of the console and then subtract the height of the toolbar from it.

However, that is nowhere near the flexibility of the original console. You’ve probably also noticed that as you resize the console the areas also resize, keeping their relative sizes.

Here, EditorGUILayout comes to the rescue! If we look at the reference for BeginVertical we’ll see that it returns a Rect object.

public static Rect BeginVertical(params GUILayoutOption[] options);

This is the rectangle that the vertical area will occupy. How the function knows it’s rectangle in advance will be covered below. For now we’ll just make use of this behavior to set the relative sizes of our areas.

float ratio = 0.9f;
Rect rect = EditorGUILayout.BeginVertical(GUILayout.ExpandHeight(true));
{
    EditorGUILayout.BeginVertical(GUILayout.Height(rect.height * ratio));
    GUILayout.Box("", box, GUILayout.ExpandHeight(true), GUILayout.ExpandWidth(true));
    EditorGUILayout.EndVertical();
    EditorGUILayout.BeginVertical();
    GUILayout.Box("", box, GUILayout.ExpandHeight(true), GUILayout.ExpandWidth(true));
    EditorGUILayout.EndVertical();
}
EditorGUILayout.EndVertical();

However, there is a problem – the size of the areas don’t seem to correspond to the ratio number. If we slap a Debug.Log(rect), we’ll see the following input:

What’s going on here? Well, we’ve stumbled onto our fist event.

Events

Whenever Unity calls OnGUI on your window it can be for various reasons, one such reason is painting the view, but it can also be because of user triggered events like button presses.
To find out the reason OnGUI was called you have to look into the Event class, and more particularly Event.current object. Let’s see what happens if we output the type of the event every time OnGUI is called.

        Debug.Log("Event Type " + Event.current.type);
        Debug.Log("Rectangle: " + rect);

We can see that the zero size of rect is associated with the Layout event. This is because Layout is a special type of event that seems to be called before painting the the actual window. It is associated with -Layout objects like GUILayout or EditorGUILayout and through this event the size of the BeginVertical rectangle is known before it’s inner elements have been drawn.

The problem with our setup is that during the layout phase we pass zeroes as the sub-area size, which messes up the layout and our areas don’t reflect what we’ve set up in code.
A small check should take care of this.

private Rect rect;
/*.....*/
Rect tmp = EditorGUILayout.BeginVertical(GUILayout.ExpandHeight(true));
{
    if ((Event.current.type != EventType.Layout) && (Event.current.type != EventType.used))
    {
    rect = tmp;
    }

    EditorGUILayout.BeginVertical(GUILayout.Height(rect.height * ratio));
    GUILayout.Box("", box, GUILayout.ExpandHeight(true), GUILayout.ExpandWidth(true));
    EditorGUILayout.EndVertical();
    EditorGUILayout.BeginVertical();
    GUILayout.Box("", box, GUILayout.ExpandHeight(true), GUILayout.ExpandWidth(true));
    EditorGUILayout.EndVertical();
}

Now, if you set ratio to something like 0.1 you’ll see that one ares is clearly smaller. EventType.used is also filtered away, since it creates a similar problem to Layout.

An EventType.used occurs when Event.Use() has been called on the current event, this prevents the inner UI elements from triggering actions on the same event.

Now that we have the console correctly displaying what we want, let’s examine the following snippet and look into some more events:

private Rect resizeRect = new Rect(0, 0, 0, 20);
private bool resizing = false;
private float ratio = 0.7f;
private const int MinBoxSize = 28; // I admit to eyeballing this from he original console

/*here lies the toolbar drawing code we did last time */
if (Event.current.type == EventType.MouseDown && resizeRect.Contains(Event.current.mousePosition))
{
    resizing = true;
}
else if (resizing && Event.current.type == EventType.MouseUp)
{
    resizing = false;
}

if (resizing && (Event.current.type == EventType.MouseDrag))
{
    resizeRect.y = Event.current.mousePosition.y - rect.y;
    ratio = resizeRect.y / rect.height;
    Repaint();
}

Rect tmp = EditorGUILayout.BeginVertical(GUILayout.ExpandHeight(true));
{
    if ((Event.current.type != EventType.Layout) && (Event.current.type != EventType.used))
    {
        rect = tmp;
        resizeRect.width = rect.width;
        resizeRect.y = Mathf.Clamp(rect.height * ratio, MinBoxSize, rect.height - MinBoxSize);
        ratio = (resizeRect.y) / rect.height;
    }

    EditorGUILayout.BeginVertical(GUILayout.Height(rect.height * ratio));
    GUILayout.Box("", box, GUILayout.ExpandHeight(true), GUILayout.ExpandWidth(true));
    EditorGUILayout.EndVertical();
    EditorGUILayout.BeginVertical();
    GUILayout.Box("", box, GUILayout.ExpandHeight(true), GUILayout.ExpandWidth(true));
    EditorGUILayout.EndVertical();
}
EditorGUILayout.EndVertical();

/*here lies the rest of the GUIWindow */

Here we use MouseDown, MouseUp and MouseDrag to check weather the user is clicking round the border area and trying to resize the box. Here, resizeRect is used to define the rectangle around the border line. Changing the last parameter of resizeRect’s constructor will change how easy it is to select the exact area around the border.Note that we don’t filter away the Mouse events when updating rect – otherwise the border line will hiccup when being dragged.

Notice the Repaint(); call after we resize the areas.

Calling Repaint() forces Unity to redraw the current GUIWindow. Repaint() needs to be called after the user makes any changes to the UI to ensure that it updates. Usually Layout and Repaint calls are made by IMGUI when the window is fist created, and when it’s resized or moved. OnGUI does not seem to be called periodically like Update, likely for performance reasons.

Cursor Rect

One last touch is the cursor – if you look at the original console you’ll see that the cursor changes to a vertical resize icon when you hover the border. How do they do that?
Enter cursor rectangles. Through EditorGUIUtility.AddCursorRect Unity allows you to change the cursor of the mouse over different areas of the UI. Thus we only need to add

EditorGUIUtility.AddCursorRect(resizeRect, MouseCursor.ResizeVertical);

in the bottom of the function to draw the resize cursor. We should end up with something like this:

This is a series of tips and tricks about creating editor extensions. We’re going to look at creating a custom console that looks and behaves exactly like the one in Unity, and eventually add more features to it.
You can find part 2 here

Custom Logging

Fist things first – before we draw any messages or buttons anywhere, we need to figure out how to receive log messages from code. There is a number of ways you can do that.

Log class

One solution would be to create a custom static class, that has a bunch of log routines like Log.Info(string message), Log.Warning(string message). This allows you to be very flexible with how you log your data (if you don’t like how Unity’s doing it). For instance, you can add a flag to disable stack trace on certain messages to improve logging performance. However, it will be hard to use it with existing projects that already employ the default console interface. Also, Unity’s internal messages – like exceptions and warning will still use the default console – so you’ll end up needing two windows.

public static class Log
{
    public static void Info(string message, UnityEngine.Object context, bool stackTrace = false, bool logToFile = true)
    {
        //log info
    }
}

Log Callback

A simple way around the internal message issue is to use Application.LogCallback, which more or less gives you all the info sent to the console for you to do whatever you want with it.
This works pretty well for most purposes, but the issue with Application.LogCallback is that it doesn’t really redirect the console output – it merely copies it. This means that behind the scenes, Unity is still doing all it’s console routines like getting the stack trace, and updating all it’s data structures that help with logging. That is something that you might not want if you’re going for a full console replacement. Also, the log callback doesn’t really provide you with the object that logged the message.

public static class Log
{
    static Log()
    {
        Application.logMessageReceived += DefaultLog;
    }

    void DefaultLog(string logString, string stackTrace, LogType type) 
    {
         if(type == LogType.Log)
         {
             Info(logString, null, true, true);
         }
       /*else
         ... other types here
      */
    }
    public static void Info(string message, UnityEngine.Object context, bool stackTrace = false, bool logToFile = true)
    {
        //log info
    }
}

ILogHandler

The Debug class allow it’s users to install their own log handlers in order to process the log output themselves. This is done through Debug.logger.logHandler and you need to supply a class that implements the ILogHandler interface. All console output will be redirected though the your custom class now – no messages will be logger in the console or the editor log.

Initialize On Load

Now the question is when to install your log handler class. Ideally you don’t want to explicitly add this code to every project that uses your console. Also, Unity projects don’t really have a clearly defined starting point.
You can use Script Execution Order to run something before all other scripts, but that still requires you to modify the project before you can use your custom console. Luckily, Unity provides just the thing we need – the InitializeOnLoad attribute makes the engine call a static constructor of the class, right after the scripts have been loaded.

Thus, our custom log handler becomes something like this.

[InitializeOnLoad]
public class LogHandler : ILogHandler
{
    public static ILogHandler DefaultHandler { get; private set; }
    private static LogHandler handler;

    static LogHandler()
    {
        DefaultHandler = Debug.logger.logHandler;
        handler = new LogHandler();
        Debug.logger.logHandler = handler;
    }

    public void LogException(Exception exception, UnityEngine.Object context)
    {              
        //handle exceptions here
    }

    public void LogFormat(LogType logType, UnityEngine.Object context, string format, params object[] args)
    {
        //handle log here
    }
}

As you can see we’re saving a reference to the default log handler, so we can still log messages to the original console when we need to. Also, to get the stack trace, you can use Unity’s StackTraceUtility. Keep in mind that it will give you the stack trace up to that point in code – you’ll need to do some formatting to get the correct stack trace.

The Window

The only requirement to creating a new editor window seems to be that you need to subclass EditorWindow class, and of course, draw something in the OnGUI method. Showing the window can be done in a number of ways, mostly involving calling EditorWindow.Show() on your window class.

However, if you want to open your window using a menu entry in Unity you need to provide a static method with a MenuItem attribute attached to it.

[MenuItem("Window/CustomConsole")]
public static void Show()
{
}

If you open the Window menu in the Editor you will find the new menu entry we’ve just added. Naturally, you can add as many menu entries for the same window as you like – each can set some specific parameters, or load data before showing the window.

Since every EditorWindow is in fact a ScriptableObject (we’ll get into that later), a window should be created using the CreateInstance method. Thus, it is not recommended to have any initialization in the constructor of the window class, unless you really know what you are doing (and let’s face it – none of us really do).

[MenuItem("Window/CustomConsole")]
public static void Show()
{
    var myWindow = ScriptableObject.CreateInstance();
    myWindow.ShowWindow();
}

The initialization code for an editor window is best put in the OnEnable. The window object will also receive callbacks for OnGUI, Update, OnDisable (when you close the window) and OnDestroy, which seems to be called right after you close the window. (At the moment of writing this I still haven’t found a way to keep the window alive after you close it)
To help you with some of the bookkeeping Unity provides a static method – EditorWindow.GetWindow. When no window of that type exists, GetWindow creates a new one and shows it, otherwise it returns a reference to the existing window.

[MenuItem("Window/CustomConsole")]
public static void Show()
{
    EditorWindow.GetWindow("Console");
}

Styles

Now, that we have our window in place, we want it to look exactly like the original console. In general it’s useful to have your editor extensions emulate the original look and feel of the editor – it makes it easier to work with them and they’re more visually pleasing. This is one aspect, where Unity definitely falls short – it does provide a limited list of styles that can be used directly, but that quickly becomes insufficient as you try to do anything beyond the basic UI.

EditorStyles

EditorStyles is a static object that provides a reference to a lot of styles that are used in drawing the editor. These can be quickly incorporated in your custom gui, but they don’t provide you with the whole picture.

The Editor Skin

The default editor skin holds all the styles that are used in drawing the editor, “hidden” in the Custom Styles list.
In order to figure out what we can use from the editor styles, or even reference them we need to access the custom style data. The best way to access the custom styles would be to simply save the editor skin in the assets(thus, extracting it from the hidden editor assets), which can be achieved with the snippet below. Note that you can also save GUI.skin – it’s the same skin, but you need to access in in the OnGUI method.

[MenuItem("Assets/Editor Skin")]
static public void SaveEditorSkin()
{
    GUISkin skin = Instantiate(EditorGUIUtility.GetBuiltinSkin(EditorSkin.Inspector));
    AssetDatabase.CreateAsset(skin, "Assets/EditorSkin.guiskin");
}

When you select the corresponding menu item, you’ll find the editor skin in your assets folder. You’ll find that there are quite a lot of custom styles used in the editor, luckily the styles we need most are close to the top.
In order to access a certain style during runtime you can simply call GUI.skin.GetStyle("CN EntryBackEven"). I’ve found that styles like EditorStyles.toolbar and GUI.skin.GetStyle("Toolbar") seem to be interchangable, so you can choose to use one or the other.
Now that we have our styles in place, we can use them do draw some of the basic elements of our console in the next part.