2D virtualization

Do you need to develop a UI grid component displaying column-aligned cells for rows bound to specific items that the client would provide? Do yourself a favor and prepare for performance improvements when you design it.

I mean: don’t just expose row item and column definition arrays that the client could populate (or data bind) with what he or she wants displayed in your grid.

I know it’s generally not a very good practice to focus on optimizing your software in its early stages, but I think this is an exception: do think about virtualization as early as possible. Both for UI itself and at the data loading level!

UI virtualization means that you’d draw just what’s visible on screen (which is good too, but many times not enough.) Data virtualization would instead go a lot further: you would simply not require the client developers to pass the entire item and column arrays that they would eventually want scrollable, and ask them instead (through a delegate protocol, event, or whatever concept the platform supports) to pass you only those items and columns that are actually needed upon drawing, given the current viewport (e.g. based on the scroll view that you have on the client side).

Of course, you can ask them for a little more than just the visible data (better in background, to just have the data handy when the end user starts to scroll towards top, bottom, left, or right) and you can cache the data for specific viewport areas until the client lets you know that data changed (again, through a delegate protocol or event), but anyway having the cache configurable (as much as possible) by the caller, to allow him or her to easily balance between runtime performance and memory usage. (But this is surely another topic, now let’s focus only on data virtualization itself.)

While data virtualization is sometimes indeed provided for item collections to be vertically scrollable in a grid (1D virtualization), I’ve rarely seen it also implemented for the horizontal direction too (i.e. 2D virtualization). So here below is a small Swift code example that could – maybe – get you started.

(This is for an infinite grid, but of course, you can define total row count and total column count values in the provider delegate protocol, and then ensure the viewport is within the total size of the output at all times in its didSet callback.)

class GridView {
    // Defines the current area actually displayed.
    var viewport = Rectangle(left: 0, top: 0, right: 0, bottom: 0) {
       didSet { updateDataProvider() }
    }

    // Configure the default row height and column width values.
    var rowHeight = 10.0 { didSet { updateDataProvider() } }
    var columnWidth = 50.0 { didSet { updateDataProvider() } }

    // Provides data for the view.
    weak var dataProvider: GridDataProvider? {
       didSet { updateDataProvider() }
    }

    // Prepare the row/column ranges that the data provider should filter on.
    private func updateDataProvider() {
        guard let dataProvider = dataProvider else { return }
        dataProvider.rowRange = 
            Range(from: Int(viewport.top/rowHeight), 
                  to: Int(viewport.bottom/rowHeight))
        dataProvider.columnRange =
            Range(from: Int(viewport.left/columnWidth), 
                  to: Int(viewport.right/columnWidth))
    }

    // "Draws" visible (filtered) cells in viewport.
    func draw() {
        guard let dataProvider = dataProvider else { return }
        for cell in dataProvider.filteredCells {
            print(cell)
        }
    }
}

struct Rectangle {
    var left, top, right, bottom: Double
}
struct Range {
    var from, to: Int
}

protocol GridDataProvider: class {
    // Input
    var rowRange: Range { get set }
    var columnRange: Range { get set }

    // Output
    var filteredCells: [GridDataCell] { get }
}

struct GridDataCell {
    var row, column: Int
    var content: String
}

// The client will act as a data provider.
class Client: GridDataProvider {
    init() { }

    // These will be updated before the view needs filtered data.
    var rowRange = Range(from: 0, to: 0)
    var columnRange = Range(from: 0, to: 0)

    // This will be called by view when it needs to display data, 
    // and we'll filter it based on input row and column ranges.
    // We'd only need to read visible data from the "database".
    var filteredCells: [GridDataCell] {
        var cells = [GridDataCell]()
        for row in rowRange.from...rowRange.to {
            for column in columnRange.from...columnRange.to {
                cells.append(GridDataCell(row: row, column: column, 
                                          content: "..."))
            }
        }
        return cells
    }

    func run() {
        // Create a grid view and tell it that we will provide data for it.
        let gridView = GridView()
        gridView.dataProvider = self
        
        // Set the viewport to the space we'd actually want to "see",
        // supposedly from a client's scroll view.
        gridView.viewport = 
            Rectangle(left: 100, top: 100, right: 160, bottom: 125)

        // Display the visible cells.
        gridView.draw()
    }
}

Client().run()

The expected output would be like this (only rows 10-12 and columns 2-3 are visible):

GridDataCell(row: 10, column: 2, content: "…")
GridDataCell(row: 10, column: 3, content: "…")
GridDataCell(row: 11, column: 2, content: "…")
GridDataCell(row: 11, column: 3, content: "…")
GridDataCell(row: 12, column: 2, content: "…")
GridDataCell(row: 12, column: 3, content: "…")

But we’re not done! Some client developers would comply because they know for sure they don’t have much data, and instead they would still want to provide all cells (for all rows and columns) at once, and would like that out “fancy” grid to display it whenever needed, without the hassle of implementing any protocol.

For them we will need to also build a good helper class (to get back their smiles):

class GridDataSource: GridDataProvider {
    init(cells: [GridDataCell]) {
        self.cells = cells
    }

    // Cache all data in memory if the client developer uses this class.
    var cells: [GridDataCell]

    var rowRange = Range(from: 0, to: 0)
    var columnRange = Range(from: 0, to: 0)

    // We'll filter the fully available data set.
    var filteredCells: [GridDataCell] {
        return cells.filter { cell in
            cell.row >= rowRange.from && cell.row <= rowRange.to &&
            cell.column >= columnRange.from && cell.column <= columnRange.to 
        }
    }    
}

// The client is way more simple when it would use a data source
// (as if it were if we didn't support virtualization!)
class Client {
    init() { }

    func run() {
        // Create a full data source.
        let dataSource = GridDataSource(cells: [
            GridDataCell(row: 11, column: 2, content: "..."),
            GridDataCell(row: 11, column: 1, content: "..."),
            GridDataCell(row: 11, column: 4, content: "..."),
            GridDataCell(row: 15, column: 3, content: "...")])

        // Then create a grid view and use that data.
        let gridView = GridView()
        gridView.dataProvider = dataSource
        
        gridView.viewport = 
            Rectangle(left: 100, top: 100, right: 160, bottom: 125)
        gridView.draw()
    }
}

Client().run()

Here is the secondary output – only one cell is displayed (as expected in our case) – as the others are on invisible rows or columns. Whoa!

GridDataCell(row: 11, column: 2, content: "…")
Advertisements
Posted in Development, Architecture, Swift | Tagged , , | Leave a comment

Medium

As you probably guessed, yes, I’m (partially) moving to Medium. (Thanks Paul for the suggestion.) Don’t worry, I will still post some work here, but – from now on – I’ll write the stories that I consider to be the most interesting only there.

Why? Simply because I like their authoring tooling better (even compared to the new WordPress editor). I don’t want to get paid or earn anything else than experience on the new platform (here in Romania I don’t even think it’s possible get payments; at least not yet, from what I’ve read.)

Posted in Miscellaneous | Leave a comment

Ramping up

When you ramp up a new project, would you rather: 
  1. assume everything is black box until details are needed or
  2. dig down, trying to understand and learn as much as possible already?

In my opinion, the best answer would depend on the type of upcoming requirements and on the quality of existing code of the project, but I’d postulate that:

  • an average value could be around 1.25 and
  • the maximum shouldn’t be higher than 1.5:
    • enthusiasm is good, but [human] memory is short and
    • it’s usually easier (and more productive) to learn by (and when) doing things.

But wait: my suggestion is probably biased due to my own way or working and experience. So: what do you think about this?

Posted in Miscellaneous | Leave a comment

Programming is…

… hard because:

brown concrete building

Fotografie de Josh Sorenson pe Pexels.com

  • to be (or become) a programmer, you don’t only need to know how to solve complex problems yourself, but you also need to be able to unleash your creativity and:
    • split such resolutions into lower and lower (and recursively lower) level steps as required by your machine, translating them into a programming language that both you and your device understand, balancing the modularity and readability (thus maintainability) of your code;
    • refactor those steps a few levels back up before you actually pass them to the machine, preparing your program to work against a development framework that both you and your device understand and support, to increase productivity and maintainability even further (or your competition releases the same app faster);
    • refactor the steps again as needed (again, often even before you actually pass them to the machine, but without breaking the toy and preserving modularity and readability as much as possible otherwise), to increase the runtime performance of the solution (face it: hardware will never be infinitely powerful – programming is engineering, not maths):
      • select the appropriate framework and programming language to use comparing the existing “wheels” (that shouldn’t be “reinvented”) to the requirements you have:
        • know as much as possible about existing (available) development technologies (features, constraints, performance indicators) before you envision the technical solution for your problem;
        • ensure your solution’s steps perfectly match the selected framework’s interface and constraints, and have backup plans to rely on when something isn’t possible considering a certain selection:
          • search for existing frameworks and analyze them in a timely manner whenever you don’t have enough knowledge about such available technologies in advance.

… yet beautiful (and well paid) because it’s hard. 🙂

Posted in Miscellaneous | Tagged , , | Leave a comment

Moq: how to VerifySet with an object that respects a condition

Sometimes you want to test that a service method sets up the correct object on another (singleton) service. For example, that an authentication service sets up a user object with the correct username to the app’s user context.

With Moq you could assume this is simple (and you would be right), but if you usually only need to verify method calls occuring on your mocks, you may have a difficult time to find the right syntax for checking the setter execution, especially as the set object also needs to respect a condition. So here is some code for the example described above (note that SetupSet is not needed here, unless we’d want to make it “globally” verifiable):

...
public class AuthenticationService
{
  ...
  public void Logon(String username)
  {
    userContext.User = new User { Username = username };
  }
}
// Arrange
var userContextMock = Mock<IUserContext>();
var authenticationService = new AuthenticationService(userContext);
// Act
authenticationService.Logon(username: "John");
// Assert
userContextMock.VerifySet(m => 
  m.User = It.Is<User>(u => u.Username == "John"));
Posted in C# | Tagged , , , | Leave a comment

Daylight may ruin your date

blur close up date dates

Fotografie de Pixabay pe Pexels.com

(No, this post doesn’t include dating advice. For such content, your best shot is probably YouTube. :-))

This is instead about Date values in Swift*. Which sometimes you might need to update by “components”. Such as by adding a specific number of days to them. And about the fact that you might – or might not – realize that, depending on which way you attempt to do it, on the actual source date value and the number of days to add to it, and finally on the target system’s settings as well (specifically when the timezone is daylight saving time enabled), the result would have a different time of day and may ruin your application!

* The concepts to be discussed apply to other development technologies too.

Moreover, date updating code could easily pass many reasonable unit tests and all system tests if they are all done with a dataset that just doesn’t fall into a specific time range. And the issue could then happen only in production, causing strange bugs your users will surely “love”. (And you might need to solve those bugs on Easter day.)

E.g. if your initial Date value was 1/1/2001 00:00 UTC, after adding 175 days (i.e. 25 weeks) to it, the result could be found to be 6/24/2001 23:00 UTC! But if you try it only with 1 and 2 days added (and even with -1), it would work just fine, always returning dates with 00:00 UTC as result.

Try it on your Mac if you don’t believe me (but as I already mentioned, you need to have your system/profile configured to use a timezone with daylight saving time enabled):

import Foundation

// Prepare a date formatter to always show dates using UTC
let dateFormatter = DateFormatter()
dateFormatter.dateStyle = .short
dateFormatter.timeStyle = .short
dateFormatter.timeZone = TimeZone.init(secondsFromGMT: 0)

// Prepare a reference date (1/1/2001)
let referenceDate = Date(timeIntervalSinceReferenceDate: 0)
print("ref:", dateFormatter.string(from: referenceDate))

// Do a first test
let date1 = Calendar.current.date(byAdding: .day, value: 1, to: referenceDate)!
print("date1", dateFormatter.string(from: date1))

// Do another test
let date2 = Calendar.current.date(byAdding: .day, value: 175, to: referenceDate)!
print("date2:", dateFormatter.string(from: date2))

If you run the above on a system with daylight saving time offset of 1 hour of a northern hemisphere country, you’d see this result, regardless of the timezone’s offset from UTC:

ref: 01/01/2001, 00:00
date1: 02/01/2001, 00:00
date2: 24/06/2001, 23:00

You guessed it: the problem is that although we have configured the DateFormatter to always use UTC timezone without daylight saving (GMT+0), the current Calendar object also has a TimeZone, and that is the current timezone. And the call on the current calendar to compute a date by adding that specific number of days seems to use the daylight saving time settings of the calendar’s timezone (i.e. those of the end user’s timezone) to internally compute the output value. And summer daylight saving time is enabled for June, but not for January nor December for a northern hemisphere DST-enabled timezone.

The fix is then very simple – the real problem here was just to do the right test and realize the issue (lesson learned: always think about and test with multiple date values, time zones, and daylight saving time settings when working with dates and times):

// Prepare generic UTC time zone
let genericTimeZone = TimeZone.init(secondsFromGMT: 0)!
...
dateFormatter.timeZone = genericTimeZone
...

// Prepare a generic UTC-based calendar for computing dates by difference
var genericCalendar = Calendar(identifier: .gregorian)
genericCalendar.timeZone = genericTimeZone

// Do a first test
let date1 = genericCalendar.date(byAdding: .day, value: 1, to: referenceDate)!
print("date1:", dateFormatter.string(from: date1))

// Do another test
let date2 = genericCalendar.date(byAdding: .day, value: 175, to: referenceDate)!
print("date2:", dateFormatter.string(from: date2))
ref: 01/01/2001, 00:00
date1: 02/01/2001, 00:00
date2: 25/06/2001, 00:00
Posted in Swift | Tagged , , , , , | Leave a comment

Swift optionals can save the day

Although generics in Swift are somehow limited at this time, there is a nice variance-related feature available when using optionals: you can assign a non-optional to an optional variable, an array of non-optionals to an array of optionals (these two are classics), but similarly you can use a function with a non-optional argument where a closure with optional argument is expected:

// Var
let i: Int? = 1

// Array
let ints: [Int?] = [1, 2]

// Func
func funcWithOptionalInt(i: Int?) {
    print(i)
}
let funcWithInt: (Int) -> Void = funcWithOptionalInt
funcWithInt(1)

This prints out “Optional(1)”, of course. But why do I think this is a such a great feature? Because this thingy can be used in a common situation like this:

Let’s assume that we need to perform an action repeatedly, and the first time it should be performed immediately, i.e. before returning from the current call. Like begin and repeat:

import Foundation

// Support for running this in a playground:
import PlaygroundSupport
PlaygroundPage.current.needsIndefiniteExecution = true

func execute(_: Timer) {
    print("Done")
}
execute() // This won't compile: execute requires a Timer argument.
let _ = Timer.scheduledTimer(withTimeInterval: 0.5, repeats: true, 
                                                    block: execute)

As you can see, we just cannot run the execute function one time “manually”, i.e. before scheduling the timer on it. There are a few standard solutions for this, of course, such as to define another execute method without an argument, and call it from the one with the Timer argument; or to call a fire() method immediately after the Timer is initialized.

But I didn’t like any of them too much (actually I’d really hate using the second one.)

And – because optionals work as described above – it seems there is a nicer way to solve the issue: to define the execute function as accepting an optional Timer argument with a default value of nil. That’s all, now everything would work, including that initial execute call, as the block closure required by scheduledTimer accepts any function with an argument of type Timer or an optional Timer! (And that default value helps us when there is no Timer object available yet to be passed to the function at all):

func execute(_: Timer? = nil) {
    print("Done")
}
execute()
let _ = Timer.scheduledTimer(withTimeInterval: 0.5, repeats: true,
                                                    block: execute)

 

Posted in Swift | Tagged , , , | Leave a comment